Apr 28 20:10:49.465379 np0000171185 systemd[1]: Started devstack@neutron-ovn-maintenance-worker.service - Devstack devstack@neutron-ovn-maintenance-worker.service.
Apr 28 20:10:49.804144 np0000171185 neutron-ovn-maintenance-worker[105714]: /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:41: EventletDeprecationWarning:
Apr 28 20:10:49.804144 np0000171185 neutron-ovn-maintenance-worker[105714]: Eventlet is deprecated. It is currently being maintained in bugfix mode, and
Apr 28 20:10:49.804144 np0000171185 neutron-ovn-maintenance-worker[105714]: we strongly recommend against using it for new projects.
Apr 28 20:10:49.804144 np0000171185 neutron-ovn-maintenance-worker[105714]: If you are already using Eventlet, we recommend migrating to a different
Apr 28 20:10:49.804144 np0000171185 neutron-ovn-maintenance-worker[105714]: framework.  For more detail see
Apr 28 20:10:49.804144 np0000171185 neutron-ovn-maintenance-worker[105714]: https://eventlet.readthedocs.io/en/latest/asyncio/migration.html
Apr 28 20:10:49.804144 np0000171185 neutron-ovn-maintenance-worker[105714]:   import eventlet
Apr 28 20:10:50.841032 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.common.config [-] Logging enabled!
Apr 28 20:10:50.841792 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-ovn-maintenance-worker version 28.1.0.dev161
Apr 28 20:10:50.841792 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-ovn-maintenance-worker --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini {{(pid=105714) setup_logging /opt/stack/neutron/neutron/common/config.py:124}}
Apr 28 20:10:50.841792 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.server.ovn_maintenance [-] OVN maintenance process starting...
Apr 28 20:10:50.841792 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "plugin-directory" by "neutron_lib.plugins.directory._create_plugin_directory" {{(pid=105714) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
Apr 28 20:10:50.842000 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" acquired by "neutron_lib.plugins.directory._create_plugin_directory" :: waited 0.000s {{(pid=105714) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
Apr 28 20:10:50.842037 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" "released" by "neutron_lib.plugins.directory._create_plugin_directory" :: held 0.000s {{(pid=105714) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
Apr 28 20:10:50.842075 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "manager" by "neutron.manager.NeutronManager._create_instance" {{(pid=105714) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
Apr 28 20:10:50.842156 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG oslo_concurrency.lockutils [-] Lock "manager" acquired by "neutron.manager.NeutronManager._create_instance" :: waited 0.000s {{(pid=105714) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
Apr 28 20:10:50.842273 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.manager [-] Loading core plugin: ml2
Apr 28 20:10:51.114972 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.115116 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.115116 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.securitygroup.SecurityGroup'>> rbac-policy before_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.115706 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.115799 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.115799 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_group.AddressGroup'>> rbac-policy before_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.149114 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.149334 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.149334 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.network.Network'>> rbac-policy before_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.252430 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.252430 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.252430 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.qos.policy.QosPolicy'>> rbac-policy before_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.286249 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.286383 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.286383 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.subnetpool.SubnetPool'>> rbac-policy before_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.309957 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.309957 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.310129 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method RbacNeutronDbObjectMixin.validate_rbac_policy_change of <class 'neutron.objects.address_scope.AddressScope'>> rbac-policy before_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.359950 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _update_segment_host_mapping_for_agent at 0x7c6b09c3a0c0> agent after_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.360046 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _update_segment_host_mapping_for_agent at 0x7c6b09c3a0c0> agent after_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.360083 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _add_segment_host_mapping_for_segment at 0x7c6b09c3a160> segment precommit_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.360151 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _delete_segments_for_network at 0x7c6b09c3a200> network precommit_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.360391 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function _prevent_segment_delete_with_port_bound at 0x7c6b09c3b420> segment before_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.412949 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._before_network_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c6b0b3af5f0>> network before_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.413120 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <function DVRDbMixin._delete_mac_associated_with_agent at 0x7c6b09c17c40> agent before_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.413120 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c6b0b3af5f0>> network before_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.413120 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c6b0b3af5f0>> port before_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.413321 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupDbMixin._ensure_default_security_group_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c6b0b3af5f0>> port before_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.413321 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c6b0b3af5f0>> segment precommit_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.413321 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c6b0b3af5f0>> segment precommit_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.413483 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c6b0b3af5f0>> segment after_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.413483 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._handle_segment_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c6b0b3af5f0>> segment after_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.413483 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._network_delete_after_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c6b0b3af5f0>> network after_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.413744 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._network_delete_precommit_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c6b0b3af5f0>> network precommit_delete 0, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.413744 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._port_provisioned of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c6b0b3af5f0>> port provisioning_complete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.413744 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._process_ext_policy_create of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c6b0b3af5f0>> rbac-policy before_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.413744 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._process_ext_policy_delete of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c6b0b3af5f0>> rbac-policy after_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.414014 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._retry_binding_revived_agents of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c6b0b3af5f0>> agent after_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.414014 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._subnet_delete_after_delete_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c6b0b3af5f0>> subnet after_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.414014 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin._subnet_delete_precommit_handler of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c6b0b3af5f0>> subnet precommit_delete 0, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.414014 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._validate_ext_not_in_use_by_project of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c6b0b3af5f0>> rbac-policy before_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.414014 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method External_net_db_mixin._validate_ext_not_in_use_by_project of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c6b0b3af5f0>> rbac-policy before_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.414366 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method DhcpAgentSchedulerDbMixin.auto_schedule_new_network_segments of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c6b0b3af5f0>> segment_host_mapping after_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.414366 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Ml2Plugin.delete_agent_notified of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c6b0b3af5f0>> agent after_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.414366 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c6b0b3af5f0>> rbac-policy before_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.414366 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c6b0b3af5f0>> rbac-policy before_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.414366 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NeutronDbPluginV2.validate_network_rbac_policy_change of <neutron.plugins.ml2.plugin.Ml2Plugin object at 0x7c6b0b3af5f0>> rbac-policy before_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.414738 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: network configured {{(pid=105714) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Apr 28 20:10:51.414833 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: port configured {{(pid=105714) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Apr 28 20:10:51.414833 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnet configured {{(pid=105714) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Apr 28 20:10:51.414955 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnetpool configured {{(pid=105714) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Apr 28 20:10:51.414955 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group configured {{(pid=105714) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Apr 28 20:10:51.414955 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group_rule configured {{(pid=105714) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Apr 28 20:10:51.415158 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.managers [-] Configured type driver names: ['local', 'flat', 'vlan', 'geneve']
Apr 28 20:10:51.415839 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.drivers.type_flat [-] Allowable flat physical_network names: ['public']
Apr 28 20:10:51.416408 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.drivers.type_local [-] ML2 LocalTypeDriver initialization complete
Apr 28 20:10:51.416972 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.drivers.type_vlan [-] Network VLAN ranges: OrderedDict({'public': [(1, 4094)]})
Apr 28 20:10:51.417112 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.managers [-] Loaded type driver names: ['flat', 'geneve', 'local', 'vlan']
Apr 28 20:10:51.417195 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.managers [-] Registered types: dict_keys(['flat', 'geneve', 'local', 'vlan'])
Apr 28 20:10:51.417268 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.managers [-] Project network_types: ['geneve']
Apr 28 20:10:51.417333 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.managers [-] Configured extension driver names: ['port_security', 'qos']
Apr 28 20:10:51.417955 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.managers [-] Loaded extension driver names: ['port_security', 'qos']
Apr 28 20:10:51.417955 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.managers [-] Registered extension drivers: ['port_security', 'qos']
Apr 28 20:10:51.417955 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.managers [-] Configured mechanism driver names: ['ovn']
Apr 28 20:10:51.453762 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.managers [-] Loaded mechanism driver names: ['ovn']
Apr 28 20:10:51.453884 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.managers [-] Registered mechanism drivers: ['ovn']
Apr 28 20:10:51.453884 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.managers [-] No mechanism drivers provide segment reachability information for agent scheduling.
Apr 28 20:10:51.454457 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Notifier._send_nova_notification of <neutron.notifiers.nova.Notifier object at 0x7c6b0996b9b0>> floatingip before_response 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.454543 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Notifier._send_nova_notification of <neutron.notifiers.nova.Notifier object at 0x7c6b0996b9b0>> port before_response 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:51.476810 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'flat'
Apr 28 20:10:51.476865 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.drivers.type_flat [-] ML2 FlatTypeDriver initialization complete
Apr 28 20:10:51.476907 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'geneve'
Apr 28 20:10:51.477032 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.drivers.type_tunnel [-] geneve ID ranges: [(1, 65536)]
Apr 28 20:10:51.521132 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO dbcounter [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Registered counter for database neutron
Apr 28 20:10:51.536245 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG oslo_db.sqlalchemy.engines [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105714) _check_effective_sql_mode /opt/stack/data/venv/lib/python3.12/site-packages/oslo_db/sqlalchemy/engines.py:325}}
Apr 28 20:10:51.938438 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG dbcounter [-] [105714] Writer thread running {{(pid=105714) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:102}}
Apr 28 20:10:52.143499 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.managers [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Initializing driver for type 'local'
Apr 28 20:10:52.143800 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.managers [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Initializing driver for type 'vlan'
Apr 28 20:10:52.255670 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.drivers.type_vlan [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] VlanTypeDriver initialization complete
Apr 28 20:10:52.255870 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.managers [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Initializing extension driver 'port_security'
Apr 28 20:10:52.255870 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.extensions.port_security [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] PortSecurityExtensionDriver initialization complete
Apr 28 20:10:52.256003 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.managers [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Initializing extension driver 'qos'
Apr 28 20:10:52.256003 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.plugins.ml2.extensions.qos [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] QosExtensionDriver initialization complete {{(pid=105714) initialize /opt/stack/neutron/neutron/plugins/ml2/extensions/qos.py:31}}
Apr 28 20:10:52.256131 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.managers [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Initializing mechanism driver 'ovn'
Apr 28 20:10:52.256131 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Starting OVNMechanismDriver
Apr 28 20:10:52.257567 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OVNMechanismDriver.pre_fork_initialize of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c6b09970c50>> process before_spawn 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.257664 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OVNMechanismDriver.post_fork_initialize of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c6b09970c50>> process after_init 55550000, True {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.257760 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OVNMechanismDriver._add_segment_host_mapping_for_segment of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c6b09970c50>> segment after_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.257876 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OVNMechanismDriver.create_segment_provnet_port of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c6b09970c50>> segment after_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.257954 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OVNMechanismDriver.delete_segment_provnet_port of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c6b09970c50>> segment after_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.258023 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OVNMechanismDriver._validate_allowed_address_pairs of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c6b09970c50>> allowed_address_pair before_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.258116 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OVNMechanismDriver._create_security_group_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c6b09970c50>> security_group precommit_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.258188 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OVNMechanismDriver._update_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c6b09970c50>> security_group after_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.258256 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OVNMechanismDriver._create_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c6b09970c50>> security_group after_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.258353 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OVNMechanismDriver._delete_security_group_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c6b09970c50>> security_group precommit_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.258426 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OVNMechanismDriver._delete_security_group of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c6b09970c50>> security_group after_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.258490 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OVNMechanismDriver._create_sg_rule_precommit of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c6b09970c50>> security_group_rule precommit_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.258581 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OVNMechanismDriver._process_sg_rule_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c6b09970c50>> security_group_rule after_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.258662 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OVNMechanismDriver._process_sg_rule_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c6b09970c50>> security_group_rule before_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.258771 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c6b09970c50>> address_group after_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.258844 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c6b09970c50>> address_group after_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.258930 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OVNMechanismDriver._process_ag_notification of <neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver object at 0x7c6b09970c50>> address_group after_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.259085 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method DriverBase._register of <neutron.services.qos.drivers.ovn.driver.OVNQosDriver object at 0x7c6b0960a3f0>> qos_plugin after_init 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.259225 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OVNTrunkDriver.register of <neutron.services.trunk.drivers.ovn.trunk_driver.OVNTrunkDriver object at 0x7c6b095e07d0>> trunk_plugin after_init 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.259405 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method DriverBase._register of <neutron.services.logapi.drivers.ovn.driver.OVNDriver object at 0x7c6b082132f0>> logging-plugin after_init 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.259798 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.services.logapi.drivers.ovn.driver [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] OVN logging driver registered
Apr 28 20:10:52.260994 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.plugins.ml2.plugin [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] neutron.plugins.ml2.plugin.Ml2Plugin method _start_rpc_notifiers called with arguments () {} {{(pid=105714) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}}
Apr 28 20:10:52.261204 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c6b08213080>> port after_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.261289 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c6b08213080>> port after_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.261375 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c6b08213080>> port after_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.261454 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c6b08213110>> subnet after_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.261533 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c6b08213110>> subnet after_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.261609 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c6b08213110>> subnet after_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.261686 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c6b08213170>> network after_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.261778 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c6b08213170>> network after_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.261854 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c6b08213170>> network after_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.261929 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c6b082131a0>> security_group after_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.262004 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c6b082131a0>> security_group after_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.262078 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c6b082131a0>> security_group after_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.262153 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c6b08213140>> security_group_rule after_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.262228 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c6b08213140>> security_group_rule after_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.262304 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c6b08213140>> security_group_rule after_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.262380 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c6b08213230>> address_group after_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.262455 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c6b08213230>> address_group after_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.262528 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method _ObjectChangeHandler.handle_event of <neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler object at 0x7c6b08213230>> address_group after_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.262605 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.plugins.ml2.ovo_rpc [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ML2 OVO RPC backend initialized. {{(pid=105714) __init__ /opt/stack/neutron/neutron/plugins/ml2/ovo_rpc.py:117}}
Apr 28 20:10:52.263254 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.quota [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded quota_driver: <neutron.db.quota.driver_nolock.DbQuotaNoLockDriver object at 0x7c6b08213260>.
Apr 28 20:10:52.263370 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.plugin [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Modular L2 Plugin initialization complete
Apr 28 20:10:52.263516 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading services supported by the core plugin {{(pid=105714) _load_services_from_core_plugin /opt/stack/neutron/neutron/manager.py:170}}
Apr 28 20:10:52.263627 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.plugins.ml2.managers [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Got port-security extension from driver 'port_security'
Apr 28 20:10:52.263901 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading service plugins: ['ovn-router', 'auto_allocate', 'tag', 'timestamp', 'network_ip_availability', 'flavors', 'revisions'] {{(pid=105714) _load_service_plugins /opt/stack/neutron/neutron/manager.py:195}}
Apr 28 20:10:52.263986 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading Plugin: ovn-router
Apr 28 20:10:52.271006 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._delete_router_remove_external_gateways of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7c6b08211910>> router before_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.271137 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <function L3RpcNotifierMixin._notify_gateway_port_ip_changed at 0x7c6b0988ade0> port after_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.271205 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <function L3RpcNotifierMixin._notify_routers_callback at 0x7c6b0988aca0> port after_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.271337 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <function L3RpcNotifierMixin._notify_subnet_gateway_ip_update at 0x7c6b0988ad40> subnet after_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.271418 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <function L3RpcNotifierMixin._notify_subnetpool_address_scope_update at 0x7c6b0988ae80> subnetpool_address_scope after_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.271546 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <function OVNL3RouterPlugin._port_update at 0x7c6b07e171a0> port before_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.271620 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <function OVNL3RouterPlugin._port_update at 0x7c6b07e171a0> port after_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.271758 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method L3_NAT_dbonly_mixin._precommit_delete_port_callback of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7c6b08211910>> port precommit_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.271871 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <function L3_NAT_dbonly_mixin._prevent_internal_ip_change_for_fip at 0x7c6b09a6ed40> port before_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.271981 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <function L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback at 0x7c6b09a6eca0> port before_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.272101 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method RouterAvailabilityZoneMixin._process_az_request of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7c6b08211910>> router precommit_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.272195 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._process_bfd_ecmp_create of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7c6b08211910>> router precommit_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.272302 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method ExtraGatewaysDbOnlyMixin._process_bfd_ecmp_update of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7c6b08211910>> router precommit_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.272468 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <function OVNL3RouterPlugin._subnet_update at 0x7c6b07e17100> subnet after_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.273139 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.quota.resource_registry [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Tracking information for resource: router configured {{(pid=105714) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Apr 28 20:10:52.273220 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.quota.resource_registry [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Tracking information for resource: floatingip configured {{(pid=105714) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:203}}
Apr 28 20:10:52.273294 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.services.ovn_l3.plugin [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Starting OVNL3RouterPlugin
Apr 28 20:10:52.273523 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OVNPortForwarding.register of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7c6b08211430>> port_forwarding_plugin after_init 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.273708 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method DriverController._check_router_request of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x7c6b08212450>> router before_create 55550100, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.273814 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method DriverController._clear_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x7c6b08212450>> router precommit_delete 55550100, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.273923 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method DriverController._set_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x7c6b08212450>> router precommit_create 55550100, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.274025 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method DriverController._update_router_provider of <neutron.services.ovn_l3.service_providers.driver_controller.DriverController object at 0x7c6b08212450>> router precommit_update 55550100, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.276448 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.services.provider_configuration [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Service providers = [] {{(pid=105714) parse_service_provider_opt /opt/stack/neutron/neutron/services/provider_configuration.py:170}}
Apr 28 20:10:52.276517 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.db.servicetype_db [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Adding provider configuration for service L3_ROUTER_NAT {{(pid=105714) add_provider_configuration /opt/stack/neutron/neutron/db/servicetype_db.py:42}}
Apr 28 20:10:52.277590 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OVN_L3_HA_db_mixin._precommit_router_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c6b08b79f10>> router precommit_create 55549900, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.277699 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OvnDriver._process_add_router_interface of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c6b08b79f10>> router_interface after_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.277819 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OvnDriver._process_floatingip_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c6b08b79f10>> floatingip after_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.277914 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OvnDriver._process_floatingip_create_update_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c6b08b79f10>> floatingip precommit_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.278145 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OvnDriver._process_floatingip_create_update_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c6b08b79f10>> floatingip precommit_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.278145 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OvnDriver._process_floatingip_delete of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c6b08b79f10>> floatingip after_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.278261 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OvnDriver._process_floatingip_status_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c6b08b79f10>> floatingip after_status_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.278324 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OvnDriver._process_floatingip_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c6b08b79f10>> floatingip after_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.278394 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OvnDriver._process_portforwarding_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c6b08b79f10>> port_forwarding after_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.278504 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OvnDriver._process_remove_router_interface of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c6b08b79f10>> router_interface after_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.278608 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OvnDriver._process_router_create of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c6b08b79f10>> router after_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.278728 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OvnDriver._process_router_create_precommit of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c6b08b79f10>> router precommit_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.278833 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OvnDriver._process_router_delete of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c6b08b79f10>> router after_delete 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.278958 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OvnDriver._process_router_update of <neutron.services.ovn_l3.service_providers.ovn.OvnDriver object at 0x7c6b08b79f10>> router after_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.279070 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.services.service_base [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded 'neutron.services.ovn_l3.service_providers.ovn.OvnDriver' provider for service L3_ROUTER_NAT {{(pid=105714) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}}
Apr 28 20:10:52.279205 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <bound method OVNL3RouterPlugin._post_fork_initialize of <neutron.services.ovn_l3.plugin.OVNL3RouterPlugin object at 0x7c6b08211910>> process after_init 55550001, True {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.279332 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Successfully loaded L3_ROUTER_NAT plugin. Description: L3 Router Service Plugin for basic L3 forwarding using OVN {{(pid=105714) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Apr 28 20:10:52.279436 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading Plugin: auto_allocate
Apr 28 20:10:52.280210 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <function _ensure_external_network_default_value_callback at 0x7c6b07e05580> network precommit_update 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.280290 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron_lib.callbacks.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Subscribe: <function _ensure_external_network_default_value_callback at 0x7c6b07e05580> network precommit_create 55550000, False {{(pid=105714) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.280563 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Successfully loaded auto-allocated-topology plugin. Description: Auto Allocated Topology - aka get me a network. {{(pid=105714) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Apr 28 20:10:52.280630 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading Plugin: tag
Apr 28 20:10:52.283083 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Successfully loaded TAG plugin. Description: Tag support {{(pid=105714) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Apr 28 20:10:52.283162 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading Plugin: timestamp
Apr 28 20:10:52.284206 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Successfully loaded timestamp plugin. Description: Adds timestamps to Neutron resources with standard attributes {{(pid=105714) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Apr 28 20:10:52.284302 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading Plugin: network_ip_availability
Apr 28 20:10:52.285515 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Successfully loaded network-ip-availability plugin. Description: Provides IP availability data for each network and subnet. {{(pid=105714) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Apr 28 20:10:52.285613 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading Plugin: flavors
Apr 28 20:10:52.286407 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Successfully loaded FLAVORS plugin. Description: Neutron Flavors and Service Profiles manager plugin {{(pid=105714) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Apr 28 20:10:52.286496 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading Plugin: revisions
Apr 28 20:10:52.287332 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.manager [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Successfully loaded revision_plugin plugin. Description: Adds revision numbers to resources. {{(pid=105714) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:239}}
Apr 28 20:10:52.287457 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG oslo_concurrency.lockutils [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Lock "manager" "released" by "neutron.manager.NeutronManager._create_instance" :: held 1.445s {{(pid=105714) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
Apr 28 20:10:52.287647 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] get_extension_paths = OrderedDict({'/opt/stack/neutron/neutron/extensions': 1}) {{(pid=105714) get_extensions_path /opt/stack/neutron/neutron/api/extensions.py:640}}
Apr 28 20:10:52.287765 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Initializing extension manager.
Apr 28 20:10:52.288012 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: __init__.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.288109 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: __pycache__ {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.288209 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: address_group.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.288546 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Address group" alias="address-group" description="Support address group" updated="2020-07-15T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.288786 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: address-group
Apr 28 20:10:52.288877 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: address_scope.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.289155 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Address scope" alias="address-scope" description="Address scopes extension." updated="2015-07-26T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.289326 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: address-scope
Apr 28 20:10:52.289415 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: admin_state_down_before_update.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.289635 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Enforce Router's Admin State Down Before Update Extension" alias="router-admin-state-down-before-update" description="Ensure that the admin state of a router is DOWN (admin_state_up=False) before updating the distributed attribute" updated="2019-07-02T15:56:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.289822 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension router-admin-state-down-before-update not supported by any of loaded plugins
Apr 28 20:10:52.289905 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: agent.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.290178 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="agent" alias="agent" description="The agent management extension." updated="2013-02-03T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.290343 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: agent
Apr 28 20:10:52.290430 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: agent_resources_synced.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.290651 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Agent's Resource View Synced to Placement" alias="agent-resources-synced" description="Stores success/failure of last sync to Placement" updated="2018-12-19T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.290852 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension agent-resources-synced not supported by any of loaded plugins
Apr 28 20:10:52.290933 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: allowedaddresspairs.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.291519 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Allowed Address Pairs" alias="allowed-address-pairs" description="Provides allowed address pairs" updated="2013-07-23T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.291697 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: allowed-address-pairs
Apr 28 20:10:52.291800 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: auto_allocated_topology.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.292027 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Auto Allocated Topology Services" alias="auto-allocated-topology" description="Auto Allocated Topology Services." updated="2016-01-01T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.292212 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: auto-allocated-topology
Apr 28 20:10:52.292288 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: availability_zone.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.292567 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Availability Zone" alias="availability_zone" description="The availability zone extension." updated="2015-01-01T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.292747 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: availability_zone
Apr 28 20:10:52.292837 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: availability_zone_filter.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.293049 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Availability Zone Filter Extension" alias="availability_zone_filter" description="Add filter parameters to AvailabilityZone resource" updated="2018-06-22T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.293218 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension availability_zone_filter not supported by any of loaded plugins
Apr 28 20:10:52.293307 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: data_plane_status.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.293536 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Port data plane status extension" alias="data-plane-status" description="Expose status of underlying data plane" updated="2017-01-24T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.293725 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension data-plane-status not supported by any of loaded plugins
Apr 28 20:10:52.293818 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: default_subnetpools.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.294053 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Default Subnetpools" alias="default-subnetpools" description="Provides ability to mark and use a subnetpool as the default." updated="2016-02-18T18:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.294211 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: default-subnetpools
Apr 28 20:10:52.294299 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: dhcpagentscheduler.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.294637 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="DHCP Agent Scheduler" alias="dhcp_agent_scheduler" description="Schedule networks among dhcp agents" updated="2013-02-07T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.294843 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: dhcp_agent_scheduler
Apr 28 20:10:52.294930 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: dns.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.295165 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="DNS Integration" alias="dns-integration" description="Provides integration with DNS." updated="2015-08-15T18:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.295345 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: dns-integration
Apr 28 20:10:52.295431 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: dns_domain_ports.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.295666 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="dns_domain for ports" alias="dns-domain-ports" description="Allows the DNS domain to be specified for a network port." updated="2017-04-24T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.295845 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: dns-domain-ports
Apr 28 20:10:52.295933 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: dns_integration_domain_keywords.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.296177 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="DNS domain names with keywords allowed" alias="dns-integration-domain-keywords" description="Allows to use keywords like <project_id>, <project_name>, <user_id> and <user_name> as DNS domain name" updated="2020-06-15T18:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.296374 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: dns-integration-domain-keywords
Apr 28 20:10:52.296501 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: dvr.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.296731 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Distributed Virtual Router" alias="dvr" description="Enables configuration of Distributed Virtual Routers." updated="2014-06-1T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.296874 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension dvr not supported by any of loaded plugins
Apr 28 20:10:52.296943 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: ecmp_routes.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.297120 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Equal Cost Multipath Routing" alias="ecmp_routes" description="Allows traffic to reach thesame destination via multiple different links " updated="2020-11-26T18:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.297259 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension ecmp_routes not supported by any of loaded plugins
Apr 28 20:10:52.297330 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: empty_string_filtering.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.297504 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Empty String Filtering Extension" alias="empty-string-filtering" description="Allow filtering by attributes with empty string value" updated="2018-05-01T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.297640 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: empty-string-filtering
Apr 28 20:10:52.297725 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: expose_l3_conntrack_helper.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.297901 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Expose CT target rules for conntrack helper" alias="expose-l3-conntrack-helper" description="Expose allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.298038 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension expose-l3-conntrack-helper not supported by any of loaded plugins
Apr 28 20:10:52.298106 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: expose_port_forwarding_in_fip.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.298280 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Expose Floating IP port forwardings" alias="expose-port-forwarding-in-fip" description="Expose port forwarding fields in floating IP response" updated="2018-06-28T16:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.298411 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension expose-port-forwarding-in-fip not supported by any of loaded plugins
Apr 28 20:10:52.298479 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: external_net.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.298651 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Neutron external network" alias="external-net" description="Adds external network attribute to network resource." updated="2013-01-14T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.298799 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: external-net
Apr 28 20:10:52.298868 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: extra_dhcp_opt.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.299043 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Neutron Extra DHCP options" alias="extra_dhcp_opt" description="Extra options configuration for DHCP. For example PXE boot options to DHCP clients can be specified (e.g. tftp-server, server-ip-address, bootfile-name)" updated="2013-03-17T12:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.299175 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: extra_dhcp_opt
Apr 28 20:10:52.299242 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: extraroute.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.299412 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Neutron Extra Route" alias="extraroute" description="Extra routes configuration for L3 router" updated="2013-02-01T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.299543 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: extraroute
Apr 28 20:10:52.299611 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: extraroute_atomic.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.299819 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Atomically add/remove extra routes" alias="extraroute-atomic" description="Edit extra routes of a router on server side by atomically adding/removing extra routes" updated="2019-07-10T00:00:00+00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.299948 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: extraroute-atomic
Apr 28 20:10:52.300017 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: filter_validation.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.300205 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Filter parameters validation" alias="filter-validation" description="Provides validation on filter parameters." updated="2018-07-04T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.300347 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: filter-validation
Apr 28 20:10:52.300445 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: fip_pf_description.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.300610 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Floating IP Port Forwarding new attribute description" alias="floating-ip-port-forwarding-description" description="Add a description field to Port Forwarding rules" updated="2019-11-01T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.300756 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension floating-ip-port-forwarding-description not supported by any of loaded plugins
Apr 28 20:10:52.300836 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: fip_pf_detail.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.301039 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Floating IP port forwarding detail" alias="floating-ip-port-forwarding-detail" description="Allow list floating ip return more port forwarding data, include'id' and 'internal_port_id'" updated="2022-05-06T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.301173 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: floating-ip-port-forwarding-detail
Apr 28 20:10:52.301242 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: fip_pf_port_range.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.301414 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Floating IP Port Forwarding support port ranges" alias="floating-ip-port-forwarding-port-ranges" description="Add support to port ranges" updated="2020-07-01T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.301545 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension floating-ip-port-forwarding-port-ranges not supported by any of loaded plugins
Apr 28 20:10:52.301612 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: fip_port_details.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.301802 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Floating IP Port Details Extension" alias="fip-port-details" description="Add port_details attribute to Floating IP resource" updated="2018-04-09T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.301931 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: fip-port-details
Apr 28 20:10:52.302000 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: flavors.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.302315 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Neutron Service Flavors" alias="flavors" description="Flavor specification for Neutron advanced services." updated="2015-09-17T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.302450 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: flavors
Apr 28 20:10:52.302519 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: floating_ip_port_forwarding.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.302783 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Floating IP Port Forwarding" alias="floating-ip-port-forwarding" description="Allow port forwarding from floating IP port to an internal port" updated="2018-01-19T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.302917 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension floating-ip-port-forwarding not supported by any of loaded plugins
Apr 28 20:10:52.302991 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: floatingip_pools.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.303205 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Floating IP Pools Extension" alias="floatingip-pools" description="Provides a floating IP pools API." updated="2018-03-21T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.303334 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: floatingip-pools
Apr 28 20:10:52.303398 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: ip_allocation.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.303574 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="IP Allocation" alias="ip_allocation" description="IP allocation extension." updated="2016-06-10T23:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.303721 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension ip_allocation not supported by any of loaded plugins
Apr 28 20:10:52.303786 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: ip_substring_port_filtering.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.303960 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="IP address substring filtering" alias="ip-substring-filtering" description="Provides IP address substring filtering when listing ports" updated="2017-11-28T09:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.304091 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension ip-substring-filtering not supported by any of loaded plugins
Apr 28 20:10:52.304158 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: l2_adjacency.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.304330 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="L2 Adjacency" alias="l2_adjacency" description="Display L2 Adjacency for Neutron Networks." updated="2016-04-12T16:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.304530 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension l2_adjacency not supported by any of loaded plugins
Apr 28 20:10:52.304599 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: l3.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.304947 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Neutron L3 Router" alias="router" description="Router abstraction for basic L3 forwarding between L2 Neutron networks and access to external networks via a NAT gateway." updated="2012-07-20T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.305095 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: router
Apr 28 20:10:52.305165 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: l3_conntrack_helper.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.305644 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="L3 Conntrack helper" alias="l3-conntrack-helper" description="Allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.305801 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension l3-conntrack-helper not supported by any of loaded plugins
Apr 28 20:10:52.305872 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: l3_enable_default_route_bfd.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.306065 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Enable Default Route BFD" alias="enable-default-route-bfd" description="Enables configurable BFD behavior for default routes" updated="2023-02-27T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.306338 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: enable-default-route-bfd
Apr 28 20:10:52.306415 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: l3_enable_default_route_ecmp.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.306602 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Enable Default Route ECMP" alias="enable-default-route-ecmp" description="Enables configurable ECMP behavior for default routes" updated="2023-02-27T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.306754 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: enable-default-route-ecmp
Apr 28 20:10:52.306817 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: l3_ext_gw_mode.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.306998 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Neutron L3 Configurable external gateway mode" alias="ext-gw-mode" description="Extension of the router abstraction for specifying whether SNAT should occur on the external gateway" updated="2013-03-28T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.307124 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: ext-gw-mode
Apr 28 20:10:52.307193 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: l3_ext_ha_mode.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.307368 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="HA Router extension" alias="l3-ha" description="Adds HA capability to routers." updated="2014-04-26T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.307495 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: l3-ha
Apr 28 20:10:52.307564 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: l3_ext_ndp_proxy.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.307753 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Router support to enable NDP proxy" alias="l3-ext-ndp-proxy" description="Router extension to enable ndp proxy" updated="2020-08-25T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.307882 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension l3-ext-ndp-proxy not supported by any of loaded plugins
Apr 28 20:10:52.307950 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: l3_extra_gws.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.308123 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Neutron L3 External Gateway Multihoming" alias="external-gateway-multihoming" description="Allow multiple external gateway ports per router" updated="2023-01-18T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.308254 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: external-gateway-multihoming
Apr 28 20:10:52.308323 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: l3_flavors.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.308566 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Router Flavor Extension" alias="l3-flavors" description="Flavor support for routers." updated="2016-05-17T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.308749 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: l3-flavors
Apr 28 20:10:52.308843 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: l3_ndp_proxy.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.309152 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="name" alias="l3-ndp-proxy" description="description" updated="2021-06-29T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.309320 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension l3-ndp-proxy not supported by any of loaded plugins
Apr 28 20:10:52.309414 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: l3_port_ip_change_not_allowed.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.309626 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Prevent L3 router ports IP address change extension" alias="l3-port-ip-change-not-allowed" description="Prevent change of IP address for some L3 router ports" updated="2018-10-09T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.309807 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension l3-port-ip-change-not-allowed not supported by any of loaded plugins
Apr 28 20:10:52.309887 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: l3agentscheduler.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.310369 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="L3 Agent Scheduler" alias="l3_agent_scheduler" description="Schedule routers among l3 agents" updated="2013-02-07T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.310507 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension l3_agent_scheduler not supported by any of loaded plugins
Apr 28 20:10:52.310574 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: local_ip.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.310871 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Local IP" alias="local_ip" description="Support Node Local Virtual IP" updated="2021-07-26T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.311008 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension local_ip not supported by any of loaded plugins
Apr 28 20:10:52.311071 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: logging.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.311320 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Logging API Extension" alias="logging" description="Provides a logging API for resources such as security group and firewall." updated="2017-01-01T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.311450 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension logging not supported by any of loaded plugins
Apr 28 20:10:52.311517 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: metering.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.311925 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Neutron Metering" alias="metering" description="Neutron Metering extension." updated="2013-06-12T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.312064 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension metering not supported by any of loaded plugins
Apr 28 20:10:52.312131 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: metering_source_and_destination_fields.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.312333 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Neutron Metering with source and destination filtering" alias="metering_source_and_destination_fields" description="Neutron Metering extension that enables the use of source and destination IP prefixes to create metering label rules." updated="2020-08-21T08:12:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.312521 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension metering_source_and_destination_fields not supported by any of loaded plugins
Apr 28 20:10:52.312590 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: multiprovidernet.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.312802 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Multi Provider Network" alias="multi-provider" description="Expose mapping of virtual networks to multiple physical networks" updated="2013-06-27T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.312930 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: multi-provider
Apr 28 20:10:52.312998 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: netmtu.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.313187 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Network MTU" alias="net-mtu" description="Provides MTU attribute for a network resource." updated="2015-03-25T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.313315 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: net-mtu
Apr 28 20:10:52.313385 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: netmtu_writable.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.313560 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Network MTU (writable)" alias="net-mtu-writable" description="Provides a writable MTU attribute for a network resource." updated="2017-07-12T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.313722 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: net-mtu-writable
Apr 28 20:10:52.313803 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: network_availability_zone.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.314008 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Network Availability Zone" alias="network_availability_zone" description="Availability zone support for network." updated="2015-01-01T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.314142 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: network_availability_zone
Apr 28 20:10:52.314211 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: network_ha.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.314386 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Network HA creation flag" alias="network_ha" description="Network high availability creation flag." updated="2023-04-27T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.314519 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension network_ha not supported by any of loaded plugins
Apr 28 20:10:52.314589 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: network_ip_availability.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.314814 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Network IP Availability" alias="network-ip-availability" description="Provides IP availability data for each network and subnet." updated="2015-09-24T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.314947 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: network-ip-availability
Apr 28 20:10:52.315021 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: network_ip_availability_details.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.315215 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Network IP Availability Details Extension" alias="network-ip-availability-details" description="Add ip_availability_details attribute to network-ip-availabilities resource." updated="2025-10-31T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.315346 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: network-ip-availability-details
Apr 28 20:10:52.315415 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: network_segment_range.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.315669 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Neutron Network Segment Range" alias="network-segment-range" description="Provides support for the network segment range management" updated="2018-11-29T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.315833 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension network-segment-range not supported by any of loaded plugins
Apr 28 20:10:52.315970 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: pagination.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.316087 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Pagination support" alias="pagination" description="Extension that indicates that pagination is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.316161 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: pagination
Apr 28 20:10:52.316229 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: port_device_profile.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.316401 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Port device profile" alias="port-device-profile" description="Expose the port device profile (Cyborg)" updated="2020-12-17T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.316571 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: port-device-profile
Apr 28 20:10:52.316644 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: port_hardware_offload_type.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.316852 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Port hardware offload" alias="port-hardware-offload-type" description="Expose port hardware offload extension" updated="2023-05-09T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.316986 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension port-hardware-offload-type not supported by any of loaded plugins
Apr 28 20:10:52.317059 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: port_hint_ovs_tx_steering.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.317240 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Port hint: Open vSwitch Tx steering" alias="port-hint-ovs-tx-steering" description="Enable Port hint: Open vSwitch Tx steering" updated="2023-01-01T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.317373 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension port-hint-ovs-tx-steering not supported by any of loaded plugins
Apr 28 20:10:52.317442 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: port_hints.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.317615 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Port hints" alias="port-hints" description="Backend specific port hints to allow backend specific tuning" updated="2023-01-01T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.317783 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension port-hints not supported by any of loaded plugins
Apr 28 20:10:52.317853 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: port_mac_address_override.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.318031 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Neutron Port MAC address override" alias="port-mac-override" description="Allow overriding the MAC address of a direct-physical Port via the active binding profile" updated="2022-03-18T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.318287 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension port-mac-override not supported by any of loaded plugins
Apr 28 20:10:52.318355 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: port_mac_address_regenerate.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.318527 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Neutron Port MAC address regenerate" alias="port-mac-address-regenerate" description="Network port MAC address regenerate" updated="2018-05-03T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.318658 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: port-mac-address-regenerate
Apr 28 20:10:52.318753 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: port_numa_affinity_policy.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.318932 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Port NUMA affinity policy" alias="port-numa-affinity-policy" description="Expose the port NUMA affinity policy" updated="2020-07-08T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.319062 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: port-numa-affinity-policy
Apr 28 20:10:52.319130 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: port_numa_affinity_policy_socket.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.319300 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Port NUMA affinity policy "socket"" alias="port-numa-affinity-policy-socket" description="Adds "socket" to the supported port NUMA affinity policies" updated="2024-02-09T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.319431 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: port-numa-affinity-policy-socket
Apr 28 20:10:52.319500 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: port_resource_request.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.319687 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Port Resource Request" alias="port-resource-request" description="Expose resource request to Port" updated="2018-05-08T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.319831 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension port-resource-request not supported by any of loaded plugins
Apr 28 20:10:52.319900 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: port_resource_request_groups.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.320083 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Port Resource Request Groups" alias="port-resource-request-groups" description="Support requesting multiple groups of resources and traits from the same RP subtree in resource_request" updated="2021-08-02T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.320217 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension port-resource-request-groups not supported by any of loaded plugins
Apr 28 20:10:52.320286 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: port_trusted_vif.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.320532 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Port trusted vif" alias="port-trusted-vif" description="Expose port 'trusted' attribute in the API" updated="2024-07-10T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.320710 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension port-trusted-vif not supported by any of loaded plugins
Apr 28 20:10:52.320773 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: portbindings.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.320967 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Port Binding" alias="binding" description="Expose port bindings of a virtual port to external application" updated="2014-02-03T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.321103 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: binding
Apr 28 20:10:52.321159 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: portbindings_extended.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.321368 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Port Bindings Extended" alias="binding-extended" description="Expose port bindings of a virtual port to external application" updated="2017-07-17T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.321499 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: binding-extended
Apr 28 20:10:52.321557 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: portsecurity.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.321757 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Port Security" alias="port-security" description="Provides port security" updated="2012-07-23T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.321888 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: port-security
Apr 28 20:10:52.321960 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: project_id.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.322129 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="project_id field enabled" alias="project-id" description="Extension that indicates that project_id field is enabled." updated="2016-09-09T09:09:09-09:09" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.322185 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: project-id
Apr 28 20:10:52.322265 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: providernet.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.322463 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Provider Network" alias="provider" description="Expose mapping of virtual networks to physical networks" updated="2012-09-07T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.322598 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: provider
Apr 28 20:10:52.322654 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: qinq.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.323769 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="QinQ networks" alias="qinq" description="Expose network 'qinq' attribute in the API. This allows to configure network to allow vlan in vlan configuration using 0x8a88 ethertype (QinQ)." updated="2024-12-02T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.323769 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: qinq
Apr 28 20:10:52.323769 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: qos.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.323769 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Quality of Service" alias="qos" description="The Quality of Service extension." updated="2015-06-08T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.324012 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension qos not supported by any of loaded plugins
Apr 28 20:10:52.324012 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: qos_bw_limit_direction.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.324142 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Direction for QoS bandwidth limit rule" alias="qos-bw-limit-direction" description="Allow to configure QoS bandwidth limit rule with specific direction: ingress or egress" updated="2017-04-10T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.324276 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension qos-bw-limit-direction not supported by any of loaded plugins
Apr 28 20:10:52.324345 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: qos_bw_minimum_ingress.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.324550 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Ingress direction for QoS minimum bandwidth rule" alias="qos-bw-minimum-ingress" description="Allow to configure QoS minumum bandwidth rule with ingress direction." updated="2018-07-09T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.324701 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension qos-bw-minimum-ingress not supported by any of loaded plugins
Apr 28 20:10:52.324778 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: qos_default.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.324942 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="QoS default policy" alias="qos-default" description="Expose the QoS default policy per project" updated="2017-041-06T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.325063 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension qos-default not supported by any of loaded plugins
Apr 28 20:10:52.325131 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: qos_fip.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.325310 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Floating IP QoS" alias="qos-fip" description="The floating IP Quality of Service extension" updated="2017-07-20T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.325435 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension qos-fip not supported by any of loaded plugins
Apr 28 20:10:52.325502 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: qos_fip_network_policy.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.325689 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="QoS floating IP network policy ID" alias="qos-fip-network-policy" description="Adds a the QoS network ID to the floating IP definition" updated="2021-11-15T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.325821 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension qos-fip-network-policy not supported by any of loaded plugins
Apr 28 20:10:52.325888 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: qos_gateway_ip.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.326059 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Router gateway IP QoS" alias="qos-gateway-ip" description="The Router gateway IP Quality of Service extension" updated="2018-02-24T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.326188 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension qos-gateway-ip not supported by any of loaded plugins
Apr 28 20:10:52.326255 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: qos_port_network_policy.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.326424 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="QoS port network policy ID" alias="qos-port-network-policy" description="Adds a the QoS network ID to the port definition" updated="2019-11-01T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.326552 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension qos-port-network-policy not supported by any of loaded plugins
Apr 28 20:10:52.326619 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: qos_pps_minimum_rule.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.326821 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="QoS minimum packet rate rule" alias="qos-pps-minimum" description="Allow to configure QoS minimum packet rate rule." updated="2021-07-14T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.326950 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension qos-pps-minimum not supported by any of loaded plugins
Apr 28 20:10:52.327017 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: qos_pps_minimum_rule_alias.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.327197 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="QoS minimum packet rate rule alias" alias="qos-pps-minimum-rule-alias" description="API to enable GET, PUT and DELETE operations on QoS minimum packet rate rule without specifying policy ID" updated="2021-10-22T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.327326 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension qos-pps-minimum-rule-alias not supported by any of loaded plugins
Apr 28 20:10:52.327393 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: qos_pps_rule.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.327579 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="QoS Rule Type Packet per Second Extension" alias="qos-pps" description="Add QoS Rule Type Packet per Second" updated="2021-05-12T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.327724 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension qos-pps not supported by any of loaded plugins
Apr 28 20:10:52.327792 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: qos_rule_type_details.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.327964 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Details of QoS rule types" alias="qos-rule-type-details" description="Expose details about QoS rule types supported by loaded backend drivers" updated="2017-06-22T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.328095 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension qos-rule-type-details not supported by any of loaded plugins
Apr 28 20:10:52.328162 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: qos_rule_type_filter.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.328332 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Allow to filter the list of QoS rule types" alias="qos-rule-type-filter" description="Allows to filter the QoS rule type list adding two new flags. "all_rules" prints all implemented QoS rule types."all_supported" prints all supported QoS rule types by the loaded mechanism drivers" updated="2022-02-02T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.328461 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension qos-rule-type-filter not supported by any of loaded plugins
Apr 28 20:10:52.328588 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: qos_rules_alias.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.328820 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Quality of Service rules alias API" alias="qos-rules-alias" description="API to enable GET, PUT and DELETE operations on QoS policy rules without specifying policy ID" updated="2018-10-07T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.328955 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension qos-rules-alias not supported by any of loaded plugins
Apr 28 20:10:52.329026 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: quota_check_limit.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.329207 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Quota engine limit check" alias="quota-check-limit" description="Support for checking the resource usage before applying a new quota limit" updated="2021-09-08T16:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.329340 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: quota-check-limit
Apr 28 20:10:52.329408 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: quota_check_limit_default.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.329600 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Quota engine limit check by default" alias="quota-check-limit-default" description="By default, the Neutron quota engine checks the resource usage before applying a new quota limit" updated="2024-08-21T16:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.329758 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: quota-check-limit-default
Apr 28 20:10:52.329831 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: quotasv2.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.330211 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Quota management support" alias="quotas" description="Expose functions for quotas management per project" updated="2012-07-29T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.330420 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: quotas
Apr 28 20:10:52.330499 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: quotasv2_detail.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.330797 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Quota details management support" alias="quota_details" description="Expose functions for quotas usage statistics per project" updated="2017-02-10T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.330897 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: quota_details
Apr 28 20:10:52.330994 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: rbac.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.331491 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="RBAC Policies" alias="rbac-policies" description="Allows creation and modification of policies that control project access to resources." updated="2015-06-17T12:15:12-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.331699 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: rbac-policies
Apr 28 20:10:52.331798 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: rbac_address_group.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.332038 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Add address_group type to RBAC" alias="rbac-address-group" description="Add address_group type to network RBAC" updated="2021-01-20T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.332220 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension rbac-address-group not supported by any of loaded plugins
Apr 28 20:10:52.332311 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: rbac_address_scope.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.332576 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Add address_scope type to RBAC" alias="rbac-address-scope" description="Add address_scope type to RBAC" updated="2020-02-12T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.332769 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: rbac-address-scope
Apr 28 20:10:52.332867 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: rbac_security_groups.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.333120 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Add security_group type to network RBAC" alias="rbac-security-groups" description="Add security_group type to network RBAC" updated="2019-02-14T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.333302 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: rbac-security-groups
Apr 28 20:10:52.333405 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: rbac_subnetpool.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.333645 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Add subnetpool type to RBAC" alias="rbac-subnetpool" description="Add subnetpool type to RBAC" updated="2020-02-05T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.333848 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension rbac-subnetpool not supported by any of loaded plugins
Apr 28 20:10:52.333944 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: revisionifmatch.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.334178 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="If-Match constraints based on revision_number" alias="revision-if-match" description="Extension indicating that If-Match based on revision_number is supported." updated="2016-12-11T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.334366 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: revision-if-match
Apr 28 20:10:52.334464 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: revisions.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.334745 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Resource revision numbers" alias="standard-attr-revisions" description="This extension will display the revision number of neutron resources." updated="2016-04-11T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.334900 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: standard-attr-revisions
Apr 28 20:10:52.334972 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: router_availability_zone.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.335194 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Router Availability Zone" alias="router_availability_zone" description="Availability zone support for router." updated="2015-01-01T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.335335 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: router_availability_zone
Apr 28 20:10:52.335406 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: router_enable_snat.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.335583 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Exposes the router "enable_snat" read-only configuration flag" alias="router-enable-snat" description="Exposes the router "enable_snat" read-only configuration flag" updated="2025-09-18T18:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.335745 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: router-enable-snat
Apr 28 20:10:52.335818 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: routerservicetype.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.335997 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Router Service Type" alias="router-service-type" description="Provides router service type" updated="2013-01-29T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.336132 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension router-service-type not supported by any of loaded plugins
Apr 28 20:10:52.336202 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: security_groups_default_rules.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.336593 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Default rules for security groups" alias="security-groups-default-rules" description="Configure set of security group rules used as default rules for every new security group" updated="2022-12-19T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.336757 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: security-groups-default-rules
Apr 28 20:10:52.336829 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: security_groups_normalized_cidr.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.337024 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Normalized CIDR field for security group rules" alias="security-groups-normalized-cidr" description="Add new field with normalized remote_ip_prefix cidr in SG rule" updated="2020-07-28T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.337163 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: security-groups-normalized-cidr
Apr 28 20:10:52.337230 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: security_groups_port_filtering.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.337414 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Port filtering on security groups" alias="port-security-groups-filtering" description="Provides security groups filtering when listing ports" updated="2018-01-09T09:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.337544 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension port-security-groups-filtering not supported by any of loaded plugins
Apr 28 20:10:52.337613 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: security_groups_remote_address_group.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.337826 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Remote address group id field for security group rules" alias="security-groups-remote-address-group" description="Add new field of remote address group id in SG rules" updated="2020-08-25T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.337964 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: security-groups-remote-address-group
Apr 28 20:10:52.338021 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: security_groups_rules_belongs_to_default_sg.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.338210 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Security group rule belongs to the project's default security group" alias="security-groups-rules-belongs-to-default-sg" description="Flag to determine if the security group rule belongs to the project's default security group" updated="2023-05-23T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.338343 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: security-groups-rules-belongs-to-default-sg
Apr 28 20:10:52.338412 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: security_groups_shared_filtering.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.338587 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Security group filtering on the shared field" alias="security-groups-shared-filtering" description="Support filtering security groups on the shared field" updated="2021-10-05T09:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.338751 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: security-groups-shared-filtering
Apr 28 20:10:52.338823 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: securitygroup.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.340161 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="security-group" alias="security-group" description="The security groups extension." updated="2012-10-05T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.340314 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: security-group
Apr 28 20:10:52.340374 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: segment.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.340658 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Segment" alias="segment" description="Segments extension." updated="2016-02-24T17:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.340831 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension segment not supported by any of loaded plugins
Apr 28 20:10:52.340890 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: segments_peer_subnet_host_routes.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.341089 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Segments peer-subnet host routes" alias="segments-peer-subnet-host-routes" description="Add host routes to subnets on a routed network (segments)" updated="2018-06-12T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.341230 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension segments-peer-subnet-host-routes not supported by any of loaded plugins
Apr 28 20:10:52.341291 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: servicetype.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.341494 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Neutron Service Type Management" alias="service-type" description="API for retrieving service providers for Neutron advanced services" updated="2013-01-20T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.341625 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: service-type
Apr 28 20:10:52.341735 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: sorting.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.341952 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Sorting support" alias="sorting" description="Extension that indicates that sorting is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.342029 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: sorting
Apr 28 20:10:52.342090 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: standard_attr_segment.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.342264 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Standard Attribute Segment Extension" alias="standard-attr-segment" description="Add standard attributes to Segment resource" updated="2018-04-09T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.342399 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension standard-attr-segment not supported by any of loaded plugins
Apr 28 20:10:52.342486 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: standardattrdescription.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.342751 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="standard-attr-description" alias="standard-attr-description" description="Extension to add descriptions to standard attributes" updated="2016-02-10T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.342892 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: standard-attr-description
Apr 28 20:10:52.342949 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: stateful_security_group.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.343141 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Stateful security group" alias="stateful-security-group" description="Indicates if the security group is stateful or not" updated="2019-11-26T09:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.343277 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: stateful-security-group
Apr 28 20:10:52.343348 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: stdattrs_common.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.343500 np0000171185 neutron-ovn-maintenance-worker[105714]: WARNING neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Did not find expected name "Stdattrs_common" in /opt/stack/neutron/neutron/extensions/stdattrs_common.py
Apr 28 20:10:52.343558 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: subnet_dns_publish_fixed_ip.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.343768 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Subnet DNS publish fixed IP" alias="subnet-dns-publish-fixed-ip" description="Support choosing to publish DNS records for IPs from a subnet" updated="2019-05-24T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.343907 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: subnet-dns-publish-fixed-ip
Apr 28 20:10:52.343963 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: subnet_external_network.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.344151 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Subnet belongs to an external network" alias="subnet-external-network" description="Informs if the subnet belongs to an external network" updated="2024-02-05T18:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.344284 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: subnet-external-network
Apr 28 20:10:52.344351 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: subnet_onboard.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.344571 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Subnet Onboard" alias="subnet_onboard" description="Provides support for onboarding subnets into subnet pools" updated="2018-12-18T09:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.344840 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension subnet_onboard not supported by any of loaded plugins
Apr 28 20:10:52.344910 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: subnet_segmentid_writable.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.345101 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Subnet SegmentID (writable)" alias="subnet-segmentid-writable" description="Provides a writable segment_id attribute for a subnet resource." updated="2018-03-12T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.345237 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension subnet-segmentid-writable not supported by any of loaded plugins
Apr 28 20:10:52.345293 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: subnet_service_types.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.345484 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Subnet service types" alias="subnet-service-types" description="Provides ability to set the subnet service_types field" updated="2016-03-15T18:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.345619 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: subnet-service-types
Apr 28 20:10:52.345692 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: subnetallocation.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.345952 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Subnet Allocation" alias="subnet_allocation" description="Enables allocation of subnets from a subnet pool" updated="2015-03-30T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.346284 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: subnet_allocation
Apr 28 20:10:52.346284 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: subnetpool_prefix_ops.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.346509 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Subnet Pool Prefix Operations" alias="subnetpool-prefix-ops" description="Provides support for adjusting the prefix list of subnet pools" updated="2019-02-08T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.346812 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: subnetpool-prefix-ops
Apr 28 20:10:52.346812 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: tag_creation.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.347033 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Tag creation extension" alias="tag-creation" description="Allow to create multiple tags for a resource" updated="2024-23-07T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.347171 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: tag-creation
Apr 28 20:10:52.347241 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: tag_ports_during_bulk_creation.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.347420 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Tag Ports During Bulk Creation" alias="tag-ports-during-bulk-creation" description="Allow to tag ports during bulk creation" updated="2019-12-29T19:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.347554 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension tag-ports-during-bulk-creation not supported by any of loaded plugins
Apr 28 20:10:52.347625 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: tagging.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.348294 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Tag support for resources with standard attribute: port, subnet, subnetpool, network, security_group, router, floatingip, policy, trunk, network_segment_range" alias="standard-attr-tag" description="Enables to set tag on resources with standard attribute." updated="2017-01-01T00:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.348438 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: standard-attr-tag
Apr 28 20:10:52.348496 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: timestamp.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.348775 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Resource timestamps" alias="standard-attr-timestamp" description="Adds created_at and updated_at fields to all Neutron resources that have Neutron standard attributes." updated="2016-09-12T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.348910 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: standard-attr-timestamp
Apr 28 20:10:52.348971 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: trunk.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.349169 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Trunk Extension" alias="trunk" description="Provides support for trunk ports" updated="2016-01-01T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.349299 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension trunk not supported by any of loaded plugins
Apr 28 20:10:52.349369 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: trunk_details.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.349556 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Trunk port details" alias="trunk-details" description="Expose trunk port details" updated="2016-01-01T10:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.349696 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension trunk-details not supported by any of loaded plugins
Apr 28 20:10:52.349787 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: uplink_status_propagation.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.349968 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Uplink status propagation extension" alias="uplink-status-propagation" description="Provides ability to propagate port uplink status." updated="2018-05-31T18:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.350102 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension uplink-status-propagation not supported by any of loaded plugins
Apr 28 20:10:52.350173 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: uplink_status_propagation_updatable.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.350348 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Uplink status propagation updatable extension" alias="uplink-status-propagation-updatable" description="Provides ability to update the port uplink status." updated="2024-09-03T18:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.350480 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Extension uplink-status-propagation-updatable not supported by any of loaded plugins
Apr 28 20:10:52.350549 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loading extension file: vlantransparent.py {{(pid=105714) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:449}}
Apr 28 20:10:52.350762 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Ext name="Vlantransparent" alias="vlan-transparent" description="Provides Vlan Transparent Networks" updated="2015-03-23T09:00:00-00:00" {{(pid=105714) _check_extension /opt/stack/neutron/neutron/api/extensions.py:415}}
Apr 28 20:10:52.350899 np0000171185 neutron-ovn-maintenance-worker[105714]: INFO neutron.api.extensions [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Loaded extension: vlan-transparent
Apr 28 20:10:52.353340 np0000171185 neutron-ovn-maintenance-worker[105714]: WARNING oslo_service.backend._threading.service [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Service MaintenanceWorker is not picklable with spawn; falling back to fork. Please make the service spawn-safe to avoid this fallback.
Apr 28 20:10:52.353596 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Full set of CONF: {{(pid=105714) _load_service_manager_options /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/oslo_config_glue.py:59}}
Apr 28 20:10:52.353734 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ******************************************************************************** {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2979}}
Apr 28 20:10:52.353799 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Configuration options gathered from: {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2980}}
Apr 28 20:10:52.353905 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2981}}
Apr 28 20:10:52.354048 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2982}}
Apr 28 20:10:52.354132 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ================================================================================ {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2987}}
Apr 28 20:10:52.354296 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] agent_down_time                = 75 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.354415 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] allow_automatic_dhcp_failover  = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.354538 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] allow_bulk                     = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.354773 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] allowed_conntrack_helpers      = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.354902 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] api_extensions_path            =  {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.355032 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] api_paste_config               = api-paste.ini {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.355162 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] api_workers                    = 4 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.355286 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] auth_strategy                  = keystone {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.355411 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] base_mac                       = fa:16:3e:00:00:00 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.355552 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] bind_host                      = 0.0.0.0 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.355695 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] bind_port                      = 9696 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.355816 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] client_socket_timeout          = 900 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.355945 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] config_dir                     = [] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.356039 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] config_file                    = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.356835 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] config_source                  = [] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.356835 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] control_exchange               = neutron {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.356835 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] core_plugin                    = ml2 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.356835 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] debug                          = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.356835 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] default_availability_zones     = [] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.357150 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] default_log_levels             = ['amqp=WARN', 'boto=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.357150 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] dhcp_agent_notification        = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.357150 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] dhcp_agents_per_network        = 1 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.357300 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] dhcp_lease_duration            = 86400 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.357378 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] dhcp_load_type                 = networks {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.357503 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] dns_domain                     = openstacklocal {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.357628 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] dvr_base_mac                   = fa:16:3f:00:00:00 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.357769 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] enable_default_route_bfd       = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.358388 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] enable_default_route_ecmp      = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.358388 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] enable_dvr                     = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.358388 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] enable_new_agents              = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.358388 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] enable_services_on_agents_with_admin_state_down = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.358388 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] enable_snat_by_default         = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.358388 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] enable_traditional_dhcp        = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.358388 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] external_dns_driver            = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.358647 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] filter_validation              = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.358647 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] global_physnet_mtu             = 1430 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.358758 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] graceful_shutdown_timeout      = 60 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.358916 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] host                           = np0000171185 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.359033 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] host_dvr_for_dhcp              = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.359151 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] http_retries                   = 3 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.359275 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] instance_format                = [instance: %(uuid)s]  {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.359397 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.359518 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ipam_driver                    = internal {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.359636 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ipv6_pd_enabled                = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.359774 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] l3_ha                          = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.359900 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] l3_ha_net_cidr                 = 169.254.192.0/18 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.360023 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] l3_ha_network_physical_name    =  {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.360143 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] l3_ha_network_type             =  {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.360265 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] log_color                      = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.360386 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] log_config_append              = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.360509 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.360688 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] log_dir                        = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.360813 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] log_file                       = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.360903 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] log_options                    = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.361027 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] log_rotate_interval            = 1 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.361151 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] log_rotate_interval_type       = days {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.361273 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] log_rotation_type              = none {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.361365 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.361457 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.361580 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.361716 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.361811 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.361932 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] max_allowed_address_pair       = 10 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.362051 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] max_dns_nameservers            = 5 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.362169 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] max_header_line                = 16384 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.362287 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] max_l3_agents_per_router       = 3 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.362406 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] max_logfile_count              = 30 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.362524 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] max_logfile_size_mb            = 200 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.362646 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] max_routes                     = 30 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.362786 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] max_subnet_host_routes         = 20 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.362912 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] my_ip                          = 199.19.213.61 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.363034 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] my_ipv6                        = 2604:e100:1:0:f816:3eff:fe24:1f60 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.363153 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] network_auto_schedule          = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.363269 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] network_link_prefix            = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.363393 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] network_scheduler_driver       = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.363515 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] notify_nova_on_port_data_changes = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.363639 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] notify_nova_on_port_status_changes = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.363782 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] pagination_max_limit           = -1 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.363903 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] periodic_fuzzy_delay           = 5 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.364022 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] periodic_interval              = 40 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.364144 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] publish_errors                 = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.364264 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] rate_limit_burst               = 0 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.364387 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] rate_limit_except_level        = CRITICAL {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.364506 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] rate_limit_interval            = 0 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.364665 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] router_distributed             = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.365641 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] rpc_resources_processing_step  = 20 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.365641 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] rpc_response_max_timeout       = 600 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.365641 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] rpc_response_timeout           = 60 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.365641 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] rpc_state_report_workers       = 0 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.365641 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] rpc_workers                    = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.365641 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] send_events_interval           = 2 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.365641 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] service_plugins                = ['ovn-router'] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.365937 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] setproctitle                   = on {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.365937 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] shell_completion               = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.365937 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] state_path                     = /opt/stack/data/neutron {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.365937 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] syslog_log_facility            = LOG_USER {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.365937 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] tcp_keepidle                   = 600 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.365937 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] transport_url                  = **** {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.366151 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] use_journal                    = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.366151 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] use_json                       = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.366237 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] use_stderr                     = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.366356 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] use_syslog                     = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.366470 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] vlan_qinq                      = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.366588 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] vlan_transparent               = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.366718 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] wsgi_default_pool_size         = 100 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.366851 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] wsgi_keep_alive                = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.366970 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] wsgi_log_format                = %(client_ip)s "%(request_line)s" status: %(status_code)s  len: %(body_length)s time: %(wall_seconds).7f {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.367089 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] wsgi_server_debug              = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.367226 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_concurrency.disable_process_locking = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.367357 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_concurrency.lock_path     = /opt/stack/data/neutron/lock {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.367490 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] profiler.connection_string     = messaging:// {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.367615 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] profiler.enabled               = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.367759 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] profiler.es_doc_type           = notification {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.367881 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] profiler.es_scroll_size        = 10000 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.368007 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] profiler.es_scroll_time        = 2m {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.368129 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] profiler.filter_error_trace    = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.368253 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] profiler.hmac_keys             = **** {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.368377 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] profiler.sentinel_service_name = mymaster {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.368508 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] profiler.socket_timeout        = 0.1 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.368691 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] profiler.trace_requests        = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.369016 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] profiler.trace_sqlalchemy      = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.369016 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] profiler_jaeger.process_tags   = {} {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.369016 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] profiler_jaeger.service_name_prefix = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.369142 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] profiler_otlp.service_name_prefix = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.369250 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.369374 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.369499 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_metrics.metrics_process_name =  {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.369628 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.369765 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.369899 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_policy.enforce_new_defaults = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.370125 np0000171185 neutron-ovn-maintenance-worker[105714]: WARNING oslo_config.cfg [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Deprecated: Option "enforce_scope" from group "oslo_policy" is deprecated for removal (This configuration was added temporarily to facilitate a smooth transition to the new RBAC. OpenStack will always enforce scope checks. This configuration option is deprecated and will be removed in the 2025.2 cycle.).  Its value may be silently ignored in the future.
Apr 28 20:10:52.370252 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_policy.enforce_scope      = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.370380 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_policy.policy_default_rule = default {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.370515 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.370639 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_policy.policy_file        = /etc/neutron/policy.json {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.370789 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.370909 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.371029 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.371145 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.371267 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.371392 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_policy.remote_timeout     = 60.0 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.371525 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.371650 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] service_providers.service_provider = [] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.371805 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.auth_section              = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.371937 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.auth_type                 = password {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.372062 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.auth_url                  = https://199.19.213.61/identity {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.372185 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.cafile                    = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.372305 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.certfile                  = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.372429 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.collect_timing            = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.372550 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.default_domain_id         = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.372688 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.default_domain_name       = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.372828 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.domain_id                 = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.372947 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.domain_name               = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.373075 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.endpoint_type             = public {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.373196 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.insecure                  = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.373315 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.keyfile                   = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.373442 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.password                  = **** {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.373560 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.project_domain_id         = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.373687 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.project_domain_name       = Default {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.373828 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.project_id                = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.373955 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.project_name              = service {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.374080 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.region_name               = RegionOne {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.374203 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.split_loggers             = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.374323 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.system_scope              = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.374443 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.timeout                   = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.374564 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.trust_id                  = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.374685 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.user_domain_id            = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.374826 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.user_domain_name          = Default {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.375068 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.user_id                   = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.375068 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] nova.username                  = nova {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.375176 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] placement.auth_section         = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.375306 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] placement.auth_type            = password {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.375425 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] placement.cafile               = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.375545 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] placement.certfile             = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.375668 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] placement.collect_timing       = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.375816 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] placement.endpoint_type        = public {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.375938 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] placement.insecure             = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.376056 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] placement.keyfile              = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.376173 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] placement.region_name          = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.376296 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] placement.split_loggers        = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.376413 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] placement.timeout              = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.376539 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ironic.auth_section            = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.376709 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ironic.auth_type               = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.376832 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ironic.cafile                  = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.376951 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ironic.certfile                = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.377074 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ironic.collect_timing          = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.377193 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ironic.connect_retries         = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.377313 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ironic.connect_retry_delay     = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.377435 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ironic.enable_notifications    = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.377555 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ironic.endpoint_override       = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.377694 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ironic.insecure                = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.377815 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ironic.interface               = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.377933 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ironic.keyfile                 = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.378053 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ironic.max_version             = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.378173 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ironic.min_version             = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.378293 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ironic.region_name             = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.378413 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ironic.retriable_status_codes  = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.378532 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ironic.service_name            = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.378652 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ironic.service_type            = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.378791 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ironic.split_loggers           = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.378911 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ironic.status_code_retries     = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.379029 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ironic.status_code_retry_delay = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.379147 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ironic.timeout                 = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.379268 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ironic.valid_interfaces        = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.379387 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ironic.version                 = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.379512 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] cli_script.dry_run             = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.379645 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.379785 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.379909 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.380029 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.380153 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.380273 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.380395 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.380517 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.380666 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.380809 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.hostname = np0000171185 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.380931 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.381059 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.381184 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.381307 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.381429 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.kombu_reconnect_splay = 0.0 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.381553 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.processname = neutron-ovn-maintenance-worker {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.381690 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.381817 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.381945 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.382067 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.382190 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.382312 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.382432 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.382554 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.382692 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.382813 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.382935 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.383057 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.383178 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.383301 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.383420 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.ssl      = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.383547 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.383689 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.383812 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.383936 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.ssl_key_file =  {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.384061 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.ssl_version =  {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.384185 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_rabbit.use_queue_manager = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.384313 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_notifications.driver = [] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.384436 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_notifications.retry = -1 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.384570 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.384762 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_messaging_notifications.transport_url = **** {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.384886 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_reports.file_event_handler = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.385009 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_reports.file_event_handler_interval = 1 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.385156 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_reports.log_dir           = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.385301 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.385353 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] QUOTAS.default_quota           = -1 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.385482 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] QUOTAS.quota_driver            = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.385607 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] QUOTAS.quota_floatingip        = 50 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.385748 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] QUOTAS.quota_network           = 100 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.385868 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] QUOTAS.quota_port              = 500 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.385989 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] QUOTAS.quota_rbac_policy       = 10 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.386110 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] QUOTAS.quota_router            = 10 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.386229 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] QUOTAS.quota_security_group    = 10 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.386351 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] QUOTAS.quota_security_group_rule = 100 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.386495 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] QUOTAS.quota_subnet            = 100 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.386587 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] QUOTAS.track_quota_usage       = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.386745 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep.capabilities           = [21, 12, 1, 2, 19] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.386864 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep.group                  = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.386986 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep.helper_command         = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.387109 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep.log_daemon_traceback   = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.387235 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep.logger_name            = oslo_privsep.daemon {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.387357 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep.thread_pool_size       = 16 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.387476 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep.user                   = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.387605 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_dhcp_release.capabilities = [21, 12] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.387744 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_dhcp_release.group     = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.387859 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_dhcp_release.helper_command = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.387981 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_dhcp_release.log_daemon_traceback = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.388108 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.388227 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_dhcp_release.thread_pool_size = 16 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.388345 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_dhcp_release.user      = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.388471 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.388588 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_ovs_vsctl.group        = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.388752 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_ovs_vsctl.helper_command = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.388871 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_ovs_vsctl.log_daemon_traceback = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.388996 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_ovs_vsctl.logger_name  = oslo_privsep.daemon {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.389118 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_ovs_vsctl.thread_pool_size = 16 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.389237 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_ovs_vsctl.user         = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.389364 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_namespace.capabilities = [21] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.389482 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_namespace.group        = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.389598 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_namespace.helper_command = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.389741 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_namespace.log_daemon_traceback = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.389862 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_namespace.logger_name  = oslo_privsep.daemon {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.389984 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_namespace.thread_pool_size = 16 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.390103 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_namespace.user         = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.390227 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_conntrack.capabilities = [12] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.390344 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_conntrack.group        = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.390462 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_conntrack.helper_command = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.390582 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_conntrack.log_daemon_traceback = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.390724 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_conntrack.logger_name  = oslo_privsep.daemon {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.390847 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_conntrack.thread_pool_size = 16 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.390966 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_conntrack.user         = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.391096 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_link.capabilities      = [12, 21] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.391214 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_link.group             = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.391330 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_link.helper_command    = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.391450 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_link.log_daemon_traceback = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.391574 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_link.logger_name       = oslo_privsep.daemon {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.391711 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_link.thread_pool_size  = 16 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.391830 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] privsep_link.user              = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.391962 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] AGENT.root_helper              = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.392088 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] AGENT.root_helper_daemon       = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.392209 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] AGENT.use_helper_for_ns_read   = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.392335 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] SECURITYGROUP.enable_ipset     = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.392462 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] SECURITYGROUP.enable_security_group = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.392579 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] SECURITYGROUP.firewall_driver  = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.392774 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] SECURITYGROUP.permitted_ethertypes = [] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.392903 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] experimental.ipv6_pd_enabled   = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.393049 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ml2.extension_drivers          = ['port_security', 'qos'] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.393168 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ml2.external_network_type      = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.393298 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ml2.mechanism_drivers          = ['ovn'] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.393426 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ml2.overlay_ip_version         = 4 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.393548 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ml2.path_mtu                   = 0 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.393670 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ml2.physical_network_mtus      = {} {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.393824 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ml2.project_network_types      = ['geneve'] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.393949 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ml2.tunnelled_network_rp_name  = rp_tunnelled {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.394090 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ml2.type_drivers               = ['local', 'flat', 'vlan', 'geneve'] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.394217 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ml2_type_flat.flat_networks    = ['public'] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.394347 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ml2_type_geneve.max_header_size = 38 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.394477 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ml2_type_geneve.vni_ranges     = ['1:65536'] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.394609 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ml2_type_vlan.network_vlan_ranges = ['public'] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.394758 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] network_log.burst_limit        = 25 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.394878 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] network_log.local_output_log_base = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.395000 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] network_log.rate_limit         = 100 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.395281 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] database.asyncio_connection    = **** {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.395281 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] database.asyncio_slave_connection = **** {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.395355 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] database.backend               = sqlalchemy {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.395492 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] database.connection            = **** {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.395613 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] database.connection_debug      = 0 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.395762 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] database.connection_parameters =  {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.395886 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] database.connection_recycle_time = 3600 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.396012 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] database.connection_trace      = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.396142 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] database.db_inc_retry_interval = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.396343 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] database.db_max_retries        = 20 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.396389 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] database.db_max_retry_interval = 10 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.396494 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] database.db_retry_interval     = 1 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.396617 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] database.max_overflow          = 50 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.396780 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] database.max_pool_size         = 5 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.396913 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] database.max_retries           = 10 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.397039 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] database.mysql_sql_mode        = TRADITIONAL {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.397158 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] database.mysql_wsrep_sync_wait = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.397278 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] database.pool_timeout          = None {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.397434 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] database.retry_interval        = 10 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.397517 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] database.slave_connection      = **** {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.397644 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] database.sqlite_synchronous    = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.397789 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] database.synchronous_reader    = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.397912 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] database.use_db_reconnect      = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.398043 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.bfd_min_rx                 = 1000 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.398171 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.bfd_min_tx                 = 100 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.398293 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.bfd_mult                   = 3 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.398414 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.broadcast_arps_to_all_routers = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.398576 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.dhcp_default_lease_time    = 43200 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.398691 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.disable_ovn_dhcp_for_baremetal_ports = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.398800 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.dns_records_ovn_owned      = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.398928 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.dns_servers                = [] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.399051 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.enable_distributed_floating_ip = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.399175 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.fdb_age_threshold          = 300 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.399304 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.ha_failover_strategy       = normal {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.399433 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.live_migration_activation_strategy = rarp {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.399556 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.localnet_learn_fdb         = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.399693 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.mac_binding_age_threshold  = 300 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.399829 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.neutron_sync_mode          = log {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.399955 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.ovn_dhcp4_global_options   = {} {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.400079 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.ovn_dhcp6_global_options   = {} {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.400207 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.ovn_l3_scheduler           = leastloaded {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.400333 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.ovn_metadata_enabled       = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.400463 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.ovn_nb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.400589 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.ovn_nb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.400792 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.ovn_nb_connection          = ['ssl:199.19.213.61:6641'] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.400919 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.ovn_nb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.401046 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.ovn_router_indirect_snat   = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.401174 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.ovn_sb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.401298 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.ovn_sb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.401432 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.ovn_sb_connection          = ['ssl:199.19.213.61:6642'] {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.401549 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.ovn_sb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.401686 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.ovs_create_tap             = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.401818 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.ovsdb_connection_timeout   = 180 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.401946 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.ovsdb_log_level            = INFO {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.402075 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.ovsdb_probe_interval       = 60000 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.402197 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.ovsdb_retry_max_interval   = 180 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.402322 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.stateless_nat_enabled      = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.402451 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn.vhost_sock_dir             = /var/run/openvswitch {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.402583 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] OVS.bridge_mac_table_size      = 50000 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.402726 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] OVS.igmp_flood                 = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.402847 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] OVS.igmp_flood_reports         = True {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.402973 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] OVS.igmp_flood_unregistered    = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.403102 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] OVS.igmp_snooping_enable       = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.403224 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] OVS.ovsdb_timeout              = 10 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.403351 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn_nb_global.fdb_removal_limit = 0 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.403473 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn_nb_global.ignore_lsp_down  = False {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.403595 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ovn_nb_global.mac_binding_removal_limit = 0 {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.403691 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG cotyledon.oslo_config_glue [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] ******************************************************************************** {{(pid=105714) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3013}}
Apr 28 20:10:52.403824 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG oslo_service.backend._threading.service [None req-0fb3d690-f037-4968-982e-9f9d20c9b04b None None] Selected the multiprocessing context: <multiprocessing.context.ForkContext object at 0x7c6b0e066930> and updated it in Cotyledon ServiceManager: <multiprocessing.context.ForkContext object at 0x7c6b0e066930> {{(pid=105714) launch_service /opt/stack/data/venv/lib/python3.12/site-packages/oslo_service/backend/_threading/service.py:394}}
Apr 28 20:10:52.429827 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] Full set of CONF: {{(pid=106506) _load_service_options /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/oslo_config_glue.py:65}}
Apr 28 20:10:52.430430 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ******************************************************************************** {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2979}}
Apr 28 20:10:52.430430 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] Configuration options gathered from: {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2980}}
Apr 28 20:10:52.430430 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2981}}
Apr 28 20:10:52.430430 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2982}}
Apr 28 20:10:52.430430 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ================================================================================ {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2987}}
Apr 28 20:10:52.430903 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] agent_down_time                = 75 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.430903 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] allow_automatic_dhcp_failover  = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.430903 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] allow_bulk                     = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.431327 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] allowed_conntrack_helpers      = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.431457 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] api_extensions_path            =  {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.431553 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] api_paste_config               = api-paste.ini {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.431738 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] api_workers                    = 4 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.431937 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] auth_strategy                  = keystone {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.432032 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] base_mac                       = fa:16:3e:00:00:00 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.432272 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] bind_host                      = 0.0.0.0 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.432387 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] bind_port                      = 9696 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.432500 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] client_socket_timeout          = 900 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.432649 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] config_dir                     = [] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.432809 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] config_file                    = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.432936 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] config_source                  = [] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.433046 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] control_exchange               = neutron {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.433124 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] core_plugin                    = ml2 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.433203 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] debug                          = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.433292 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] default_availability_zones     = [] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.433467 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] default_log_levels             = ['amqp=WARN', 'boto=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.433572 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] dhcp_agent_notification        = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.433641 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] dhcp_agents_per_network        = 1 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.433731 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] dhcp_lease_duration            = 86400 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.433772 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] dhcp_load_type                 = networks {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.433854 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] dns_domain                     = openstacklocal {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.433917 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] dvr_base_mac                   = fa:16:3f:00:00:00 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.434001 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] enable_default_route_bfd       = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.434061 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] enable_default_route_ecmp      = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.434148 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] enable_dvr                     = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.434210 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] enable_new_agents              = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.434261 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] enable_services_on_agents_with_admin_state_down = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.434345 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] enable_snat_by_default         = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.434420 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] enable_traditional_dhcp        = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.434507 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] external_dns_driver            = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.434569 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] fatal_deprecations             = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.434637 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] filter_validation              = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.434698 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] global_physnet_mtu             = 1430 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.434754 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] graceful_shutdown_timeout      = 60 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.434954 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] host                           = np0000171185 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.435028 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] host_dvr_for_dhcp              = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.435100 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] http_retries                   = 3 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.435167 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] instance_format                = [instance: %(uuid)s]  {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.435232 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.435296 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ipam_driver                    = internal {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.435364 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ipv6_pd_enabled                = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.435498 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] l3_ha                          = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.435586 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_net_cidr                 = 169.254.192.0/18 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.435656 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_network_physical_name    =  {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.435757 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] l3_ha_network_type             =  {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.435825 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] log_color                      = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.435904 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] log_config_append              = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.435988 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.436058 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] log_dir                        = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.436119 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] log_file                       = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.436157 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] log_options                    = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.436226 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] log_rotate_interval            = 1 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.436298 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] log_rotate_interval_type       = days {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.436361 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] log_rotation_type              = none {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.436426 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.436465 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.436503 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.436581 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.436630 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.436776 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] max_allowed_address_pair       = 10 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.436851 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] max_dns_nameservers            = 5 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.436894 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] max_header_line                = 16384 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.436948 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] max_l3_agents_per_router       = 3 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.436993 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] max_logfile_count              = 30 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.437044 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] max_logfile_size_mb            = 200 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.437115 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] max_routes                     = 30 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.437188 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] max_subnet_host_routes         = 20 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.437258 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] my_ip                          = 199.19.213.61 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.437336 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] my_ipv6                        = 2604:e100:1:0:f816:3eff:fe24:1f60 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.437414 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] network_auto_schedule          = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.437480 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] network_link_prefix            = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.437551 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] network_scheduler_driver       = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.437616 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] notify_nova_on_port_data_changes = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.437722 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] notify_nova_on_port_status_changes = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.437795 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] pagination_max_limit           = -1 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.437918 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] periodic_fuzzy_delay           = 5 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.438034 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] periodic_interval              = 40 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.438133 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] publish_errors                 = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.438199 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_burst               = 0 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.438272 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_except_level        = CRITICAL {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.438331 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] rate_limit_interval            = 0 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.438405 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] router_distributed             = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.438465 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] rpc_resources_processing_step  = 20 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.438535 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] rpc_response_max_timeout       = 600 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.438610 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] rpc_response_timeout           = 60 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.438690 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] rpc_state_report_workers       = 0 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.438756 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] rpc_workers                    = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.438814 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] send_events_interval           = 2 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.438890 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] service_plugins                = ['ovn-router'] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.438954 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] setproctitle                   = on {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.439028 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] shell_completion               = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.439322 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] state_path                     = /opt/stack/data/neutron {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.439396 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] syslog_log_facility            = LOG_USER {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.439462 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] tcp_keepidle                   = 600 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.439527 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] transport_url                  = **** {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.439597 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] use_journal                    = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.439648 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] use_json                       = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.439727 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] use_stderr                     = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.439792 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] use_syslog                     = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.439853 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] vlan_qinq                      = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.439910 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] vlan_transparent               = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.439977 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] wsgi_default_pool_size         = 100 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.440037 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] wsgi_keep_alive                = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.440117 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] wsgi_log_format                = %(client_ip)s "%(request_line)s" status: %(status_code)s  len: %(body_length)s time: %(wall_seconds).7f {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.440178 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] wsgi_server_debug              = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:2995}}
Apr 28 20:10:52.440377 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_concurrency.disable_process_locking = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.440465 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_concurrency.lock_path     = /opt/stack/data/neutron/lock {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.440565 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] profiler.connection_string     = messaging:// {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.440657 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] profiler.enabled               = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.440770 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_doc_type           = notification {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.440844 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_scroll_size        = 10000 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.440917 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] profiler.es_scroll_time        = 2m {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.440996 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] profiler.filter_error_trace    = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.441068 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] profiler.hmac_keys             = **** {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.441156 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] profiler.sentinel_service_name = mymaster {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.441246 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] profiler.socket_timeout        = 0.1 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.441309 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] profiler.trace_requests        = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.441384 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] profiler.trace_sqlalchemy      = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.441476 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] profiler_jaeger.process_tags   = {} {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.441540 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] profiler_jaeger.service_name_prefix = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.441613 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] profiler_otlp.service_name_prefix = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.441714 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.441797 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.441865 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_process_name =  {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.441938 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.442002 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.442092 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.enforce_new_defaults = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.442139 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.enforce_scope      = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.442186 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_default_rule = default {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.442229 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.442277 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.policy_file        = /etc/neutron/policy.json {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.442329 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.442377 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.442419 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.442528 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.442528 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.442584 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_policy.remote_timeout     = 60.0 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.442622 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.442662 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] service_providers.service_provider = [] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.442734 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_section              = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.442794 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_type                 = password {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.442835 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.auth_url                  = https://199.19.213.61/identity {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.442870 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.cafile                    = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.442916 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.certfile                  = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.443280 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.collect_timing            = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.443280 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.default_domain_id         = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.443280 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.default_domain_name       = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.443280 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.domain_id                 = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.443280 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.domain_name               = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.443280 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.endpoint_type             = public {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.443280 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.insecure                  = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.443280 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.keyfile                   = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.443280 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.password                  = **** {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.443539 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.project_domain_id         = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.443539 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.project_domain_name       = Default {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.443539 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.project_id                = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.443539 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.project_name              = service {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.443539 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.region_name               = RegionOne {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.443539 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.split_loggers             = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.443734 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.system_scope              = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.443734 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.timeout                   = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.443734 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.trust_id                  = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.443734 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.user_domain_id            = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.443734 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.user_domain_name          = Default {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.443892 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.user_id                   = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.443892 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] nova.username                  = nova {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.443892 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] placement.auth_section         = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.443979 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] placement.auth_type            = password {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.444046 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] placement.cafile               = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.444083 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] placement.certfile             = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.444113 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] placement.collect_timing       = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.444142 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] placement.endpoint_type        = public {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.444177 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] placement.insecure             = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.444220 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] placement.keyfile              = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.444262 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] placement.region_name          = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.444297 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] placement.split_loggers        = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.444346 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] placement.timeout              = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.444399 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ironic.auth_section            = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.444435 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ironic.auth_type               = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.444484 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ironic.cafile                  = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.444531 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ironic.certfile                = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.444567 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ironic.collect_timing          = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.444601 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ironic.connect_retries         = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.444634 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ironic.connect_retry_delay     = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.444669 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ironic.enable_notifications    = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.444775 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ironic.endpoint_override       = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.444835 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ironic.insecure                = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.444835 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ironic.interface               = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.444899 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ironic.keyfile                 = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.444942 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ironic.max_version             = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.444942 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ironic.min_version             = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.445005 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ironic.region_name             = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.445046 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ironic.retriable_status_codes  = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.445082 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ironic.service_name            = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.445116 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ironic.service_type            = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.445149 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ironic.split_loggers           = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.445184 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ironic.status_code_retries     = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.445234 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ironic.status_code_retry_delay = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.445280 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ironic.timeout                 = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.445312 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ironic.valid_interfaces        = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.445343 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ironic.version                 = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.445392 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] cli_script.dry_run             = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.445460 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.445493 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.445557 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.445610 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.445661 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.445723 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.445755 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.445803 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.445836 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.445888 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.hostname = np0000171185 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.445931 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.445972 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.446014 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.446056 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.446098 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.kombu_reconnect_splay = 0.0 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.446140 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.processname = neutron-ovn-maintenance-worker {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.446181 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.446225 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.446262 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.446304 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.446345 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.446386 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.446428 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.446465 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.446507 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.446547 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.446588 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.446630 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.446671 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.446731 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.446770 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl      = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.446812 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.446850 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.446900 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.446935 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_key_file =  {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.446977 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.ssl_version =  {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.447013 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_rabbit.use_queue_manager = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.447074 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.driver = [] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.447113 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.retry = -1 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.447162 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.447200 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_messaging_notifications.transport_url = **** {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.447261 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.file_event_handler = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.447298 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.file_event_handler_interval = 1 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.447352 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_reports.log_dir           = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.447408 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.447476 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.default_quota           = -1 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.447527 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_driver            = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.447575 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_floatingip        = 50 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.447615 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_network           = 100 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.447665 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_port              = 500 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.447731 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_rbac_policy       = 10 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.447780 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_router            = 10 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.447818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_security_group    = 10 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.447866 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_security_group_rule = 100 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.447911 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.quota_subnet            = 100 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.447960 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] QUOTAS.track_quota_usage       = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.448021 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep.capabilities           = [21, 12, 1, 2, 19] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.448069 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep.group                  = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.448121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep.helper_command         = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.448219 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep.log_daemon_traceback   = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.448274 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep.logger_name            = oslo_privsep.daemon {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.448321 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep.thread_pool_size       = 16 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.448368 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep.user                   = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.448416 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.capabilities = [21, 12] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.448457 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.group     = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.448498 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.helper_command = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.448539 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.log_daemon_traceback = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.448588 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.448622 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.thread_pool_size = 16 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.448668 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_dhcp_release.user      = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.448760 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.448823 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.group        = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.448865 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.helper_command = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.448907 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.log_daemon_traceback = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.448948 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.logger_name  = oslo_privsep.daemon {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.448984 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.thread_pool_size = 16 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.449029 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_ovs_vsctl.user         = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.449086 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.capabilities = [21] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.449127 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.group        = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.449174 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.helper_command = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.449216 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.log_daemon_traceback = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.449257 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.logger_name  = oslo_privsep.daemon {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.449298 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.thread_pool_size = 16 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.449341 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_namespace.user         = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.449386 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.capabilities = [12] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.449427 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.group        = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.449463 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.helper_command = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.449506 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.log_daemon_traceback = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.449548 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.logger_name  = oslo_privsep.daemon {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.449589 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.thread_pool_size = 16 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.449630 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_conntrack.user         = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.449701 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.capabilities      = [12, 21] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.449746 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.group             = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.449786 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.helper_command    = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.449827 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.log_daemon_traceback = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.449865 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.logger_name       = oslo_privsep.daemon {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.449907 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.thread_pool_size  = 16 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.449948 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] privsep_link.user              = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.450005 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] AGENT.root_helper              = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.450055 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] AGENT.root_helper_daemon       = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.450117 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] AGENT.use_helper_for_ns_read   = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.450281 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.enable_ipset     = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.450376 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.enable_security_group = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.450482 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.firewall_driver  = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.450563 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] SECURITYGROUP.permitted_ethertypes = [] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.450855 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] experimental.ipv6_pd_enabled   = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.450925 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ml2.extension_drivers          = ['port_security', 'qos'] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.450970 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ml2.external_network_type      = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.451032 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ml2.mechanism_drivers          = ['ovn'] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.451073 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ml2.overlay_ip_version         = 4 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.451122 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ml2.path_mtu                   = 0 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.451159 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ml2.physical_network_mtus      = {} {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.451200 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ml2.project_network_types      = ['geneve'] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.451246 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ml2.tunnelled_network_rp_name  = rp_tunnelled {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.451305 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ml2.type_drivers               = ['local', 'flat', 'vlan', 'geneve'] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.451373 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_flat.flat_networks    = ['public'] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.451443 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_geneve.max_header_size = 38 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.451506 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_geneve.vni_ranges     = ['1:65536'] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.451573 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ml2_type_vlan.network_vlan_ranges = ['public'] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.451634 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] network_log.burst_limit        = 25 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.451699 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] network_log.local_output_log_base = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.451746 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] network_log.rate_limit         = 100 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.451812 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] database.asyncio_connection    = **** {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.451871 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] database.asyncio_slave_connection = **** {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.451920 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] database.backend               = sqlalchemy {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.451974 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] database.connection            = **** {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.452024 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] database.connection_debug      = 0 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.452067 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] database.connection_parameters =  {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.452122 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] database.connection_recycle_time = 3600 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.452165 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] database.connection_trace      = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.452205 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] database.db_inc_retry_interval = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.452251 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] database.db_max_retries        = 20 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.452297 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] database.db_max_retry_interval = 10 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.452339 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] database.db_retry_interval     = 1 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.452381 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] database.max_overflow          = 50 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.452424 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] database.max_pool_size         = 5 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.452466 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] database.max_retries           = 10 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.452517 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] database.mysql_sql_mode        = TRADITIONAL {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.452560 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] database.mysql_wsrep_sync_wait = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.452605 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] database.pool_timeout          = None {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.452650 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] database.retry_interval        = 10 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.452717 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] database.slave_connection      = **** {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.452801 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] database.sqlite_synchronous    = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.452831 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] database.synchronous_reader    = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.452872 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] database.use_db_reconnect      = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.452934 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_min_rx                 = 1000 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.452974 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_min_tx                 = 100 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.453018 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.bfd_mult                   = 3 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.453061 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.broadcast_arps_to_all_routers = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.453113 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.dhcp_default_lease_time    = 43200 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.453157 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.disable_ovn_dhcp_for_baremetal_ports = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.453200 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.dns_records_ovn_owned      = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.453245 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.dns_servers                = [] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.453294 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.enable_distributed_floating_ip = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.453327 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.fdb_age_threshold          = 300 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.453384 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.ha_failover_strategy       = normal {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.453418 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.live_migration_activation_strategy = rarp {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.453461 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.localnet_learn_fdb         = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.453502 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.mac_binding_age_threshold  = 300 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.453553 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.neutron_sync_mode          = log {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.453598 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_dhcp4_global_options   = {} {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.453648 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_dhcp6_global_options   = {} {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.453717 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_l3_scheduler           = leastloaded {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.453770 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_metadata_enabled       = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.453804 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.453846 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.453884 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_connection          = ['ssl:199.19.213.61:6641'] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.453920 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_nb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.453979 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_router_indirect_snat   = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.454016 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_ca_cert             = /opt/stack/data/CA/int-ca/ca-chain.pem {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.454049 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_certificate         = /opt/stack/data/CA/int-ca/devstack-cert.crt {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.454090 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_connection          = ['ssl:199.19.213.61:6642'] {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.454132 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovn_sb_private_key         = /opt/stack/data/CA/int-ca/private/devstack-cert.key {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.454168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovs_create_tap             = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.454202 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_connection_timeout   = 180 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.454253 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_log_level            = INFO {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.454288 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_probe_interval       = 60000 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.454322 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.ovsdb_retry_max_interval   = 180 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.454360 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.stateless_nat_enabled      = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.454423 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn.vhost_sock_dir             = /var/run/openvswitch {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.454474 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] OVS.bridge_mac_table_size      = 50000 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.454525 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood                 = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.454574 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood_reports         = True {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.454612 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_flood_unregistered    = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.454661 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] OVS.igmp_snooping_enable       = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.454724 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] OVS.ovsdb_timeout              = 10 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.454783 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.fdb_removal_limit = 0 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.454829 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.ignore_lsp_down  = False {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.454875 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ovn_nb_global.mac_binding_removal_limit = 0 {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3006}}
Apr 28 20:10:52.454919 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon.oslo_config_glue [-] ******************************************************************************** {{(pid=106506) log_opt_values /opt/stack/data/venv/lib/python3.12/site-packages/oslo_config/cfg.py:3013}}
Apr 28 20:10:52.455059 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG cotyledon._service_worker [-] Run service ServiceWrapper(0) [106506] {{(pid=106506) wait_forever /opt/stack/data/venv/lib/python3.12/site-packages/cotyledon/_service_worker.py:218}}
Apr 28 20:10:52.455753 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver.post_fork_initialize-820103', 'neutron.services.ovn_l3.plugin.OVNL3RouterPlugin._post_fork_initialize-16517081'] for process (None), after_init {{(pid=106506) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
Apr 28 20:10:52.456989 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbNbOvnIdl for MaintenanceWorker with retry
Apr 28 20:10:52.520839 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Logical_Switch.name {{(pid=106506) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Apr 28 20:10:52.521045 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Logical_Router.name {{(pid=106506) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Apr 28 20:10:52.521311 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Load_Balancer.name {{(pid=106506) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Apr 28 20:10:52.521517 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Copp.name {{(pid=106506) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 28 20:10:52.521666 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Meter.name {{(pid=106506) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 28 20:10:52.521863 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Load_Balancer_Group.name {{(pid=106506) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 28 20:10:52.522011 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index HA_Chassis_Group.name {{(pid=106506) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 28 20:10:52.522155 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Address_Set.name {{(pid=106506) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 28 20:10:52.522316 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Connection.target {{(pid=106506) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 28 20:10:52.522450 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Gateway_Chassis.name {{(pid=106506) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 28 20:10:52.522581 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Mirror.name {{(pid=106506) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 28 20:10:52.522729 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Chassis_Template_Var.chassis {{(pid=106506) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 28 20:10:52.522841 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Logical_Router_Port.name {{(pid=106506) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 28 20:10:52.522951 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Port_Group.name {{(pid=106506) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 28 20:10:52.523057 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Logical_Switch_Port.name {{(pid=106506) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 28 20:10:52.527246 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.19.213.61:6641: connecting...
Apr 28 20:10:52.534709 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.19.213.61:6641: connected
Apr 28 20:10:52.562396 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [-] Getting OvsdbSbOvnIdl for MaintenanceWorker with retry
Apr 28 20:10:52.596067 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Chassis.name {{(pid=106506) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Apr 28 20:10:52.596227 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl [-] Created lookup_table index Port_Binding.logical_port {{(pid=106506) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:87}}
Apr 28 20:10:52.596227 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Chassis_Private.name {{(pid=106506) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 28 20:10:52.596227 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl [-] Created schema index Datapath_Binding.tunnel_key {{(pid=106506) autocreate_indices /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Apr 28 20:10:52.598218 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.19.213.61:6642: connecting...
Apr 28 20:10:52.637543 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO ovsdbapp.backend.ovs_idl.vlog [-] ssl:199.19.213.61:6642: connected
Apr 28 20:10:52.674587 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_floatingip_request of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7c6b089d98b0>> floatingip precommit_update 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.674723 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_floatingip_request of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7c6b089d98b0>> floatingip precommit_delete 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.674787 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_port_has_port_forwarding of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7c6b089d98b0>> floatingip before_create 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.674845 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._check_port_has_port_forwarding of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7c6b089d98b0>> floatingip before_update 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.674926 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._process_port_request_handler of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7c6b089d98b0>> port after_update 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.674997 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method PortForwardingPlugin._process_port_request_handler of <neutron.services.portforwarding.pf_plugin.PortForwardingPlugin object at 0x7c6b089d98b0>> port precommit_delete 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.675224 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding.register-16516243'] for port_forwarding_plugin (None), after_init {{(pid=106506) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
Apr 28 20:10:52.675318 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7c6b08211430>> port_forwarding after_create 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.675318 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_lb_on_ls of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7c6b08211430>> router_interface after_create 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.675444 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7c6b08211430>> port_forwarding after_update 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.675444 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_notification of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7c6b08211430>> port_forwarding after_delete 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.675580 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method OVNPortForwarding._handle_lb_on_ls of <neutron.services.portforwarding.drivers.ovn.driver.OVNPortForwarding object at 0x7c6b08211430>> router_interface after_delete 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.675648 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
Apr 28 20:10:52.675740 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.placement [-] Starting OVNClientPlacementExtension
Apr 28 20:10:52.676136 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Plugin._prevent_segment_delete_with_subnet_associated of <neutron.services.segments.plugin.Plugin object at 0x7c6b089db9b0>> segment before_delete 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.676208 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method Plugin._validate_auto_address_subnet_delete of <neutron.services.segments.plugin.Plugin object at 0x7c6b089db9b0>> subnet precommit_delete_associations 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.676485 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_host_addition_to_aggregate of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c6b089d98e0>> segment_host_mapping after_create 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.676594 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_host_removal_from_aggregate of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c6b089d98e0>> segment_host_mapping after_delete 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.676594 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_created_or_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c6b089d98e0>> port after_create 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.676835 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_created_or_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c6b089d98e0>> port after_delete 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.676835 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_port_updated of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c6b089d98e0>> port after_update 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.676835 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_segment_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c6b089d98e0>> segment after_delete 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.676835 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_created of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c6b089d98e0>> subnet after_create 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.677055 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_deleted of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c6b089d98e0>> subnet after_delete 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.677055 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method NovaSegmentNotifier._notify_subnet_updated of <neutron.services.segments.plugin.NovaSegmentNotifier object at 0x7c6b089d98e0>> subnet after_update 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.735142 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_after_create of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x7c6b08a11d90>> subnet after_create 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.735266 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_after_delete of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x7c6b08a11d90>> subnet after_delete 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.735266 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_before_create of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x7c6b08a11d90>> subnet before_create 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.735406 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SegmentHostRoutes.host_routes_before_update of <neutron.services.segments.plugin.SegmentHostRoutes object at 0x7c6b08a11d90>> subnet before_update 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.737464 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method LoggingPlugin._clean_logs_by_resource_id of <neutron.services.logapi.logging_plugin.LoggingPlugin object at 0x7c6b089cd370>> security_group after_delete 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.737552 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method LoggingPlugin._clean_logs_by_target_id of <neutron.services.logapi.logging_plugin.LoggingPlugin object at 0x7c6b089cd370>> port after_delete 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.737713 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Publish callbacks ['neutron.services.logapi.drivers.ovn.driver.DriverBase._register-1572815'] for logging-plugin (None), after_init {{(pid=106506) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
Apr 28 20:10:52.737829 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x7c6b08a11dc0>> security_group_rule after_create 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.737887 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x7c6b08a11dc0>> security_group_rule after_update 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:10:52.737946 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: <bound method SecurityGroupRuleCallBack.handle_event of <neutron.services.logapi.common.sg_callback.SecurityGroupRuleCallBack object at 0x7c6b08a11dc0>> security_group_rule after_delete 55550000, False {{(pid=106506) subscribe /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:52}}
Apr 28 20:11:02.150291 np0000171185 neutron-ovn-maintenance-worker[105714]: DEBUG dbcounter [-] [105714] Writing DB stats neutron:SELECT=4 {{(pid=105714) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:11:02.739088 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [-] OVN-Northbound DB sync process started @ 2026-04-28 20:11:02.738627 {{(pid=106506) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:99}}
Apr 28 20:11:02.741634 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d4918b1d-620c-42ce-a161-d7da65079db7 None None] OVN-NB Sync port groups started @ 2026-04-28 20:11:02.741292 {{(pid=106506) sync_port_groups /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:132}}
Apr 28 20:11:02.762780 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writer thread running {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:102}}
Apr 28 20:11:02.776168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d4918b1d-620c-42ce-a161-d7da65079db7 None None] OVN-NB Sync port groups completed @ 2026-04-28 20:11:02.775883 {{(pid=106506) sync_port_groups /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:196}}
Apr 28 20:11:02.776381 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d4918b1d-620c-42ce-a161-d7da65079db7 None None] OVN-NB Sync networks, ports and DHCP options started @ 2026-04-28 20:11:02.776118 {{(pid=106506) sync_networks_ports_and_dhcp_opts /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1150}}
Apr 28 20:11:03.072536 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.services.segments.db [None req-d4918b1d-620c-42ce-a161-d7da65079db7 None None] neutron.services.segments.plugin.Plugin method get_segments called with arguments (<neutron_lib.context.Context object at 0x7c6b081e50d0>,) {'filters': {'network_id': ['7ff08466-af6d-4e13-bb1d-8987170a5734'], 'is_dynamic': False}} {{(pid=106506) wrapper /opt/stack/data/venv/lib/python3.12/site-packages/oslo_log/helpers.py:77}}
Apr 28 20:11:03.077261 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d4918b1d-620c-42ce-a161-d7da65079db7 None None] OVN-NB Sync metadata ports started {{(pid=106506) _sync_metadata_ports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1095}}
Apr 28 20:11:03.146293 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d4918b1d-620c-42ce-a161-d7da65079db7 None None] OVN-NB Sync metadata ports completed {{(pid=106506) _sync_metadata_ports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1147}}
Apr 28 20:11:03.146453 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d4918b1d-620c-42ce-a161-d7da65079db7 None None] OVN-NB Sync DHCP options for Neutron subnets started {{(pid=106506) _sync_subnet_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:947}}
Apr 28 20:11:03.212790 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d4918b1d-620c-42ce-a161-d7da65079db7 None None] OVN-NB Sync DHCP options for Neutron subnets completed @ 2026-04-28 20:11:03.212326 {{(pid=106506) _sync_subnet_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1020}}
Apr 28 20:11:03.213550 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
Apr 28 20:11:03.213854 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d4918b1d-620c-42ce-a161-d7da65079db7 None None] OVN-NB Sync DHCP options for Neutron ports with extra dhcp options assigned started {{(pid=106506) _sync_port_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1025}}
Apr 28 20:11:03.213954 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d4918b1d-620c-42ce-a161-d7da65079db7 None None] OVN-NB Sync DHCP options for Neutron ports with extra dhcp options assigned completed {{(pid=106506) _sync_port_dhcp_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1084}}
Apr 28 20:11:03.214109 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d4918b1d-620c-42ce-a161-d7da65079db7 None None] OVN-NB Sync networks, ports and DHCP options completed @ 2026-04-28 20:11:03.213962 {{(pid=106506) sync_networks_ports_and_dhcp_opts /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1319}}
Apr 28 20:11:03.214282 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d4918b1d-620c-42ce-a161-d7da65079db7 None None] OVN-NB Sync ACLs started @ 2026-04-28 20:11:03.214127 {{(pid=106506) sync_acls /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:239}}
Apr 28 20:11:03.258380 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d4918b1d-620c-42ce-a161-d7da65079db7 None None] OVN-NB Sync ACLs completed @ 2026-04-28 20:11:03.257989 {{(pid=106506) sync_acls /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:429}}
Apr 28 20:11:03.258380 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d4918b1d-620c-42ce-a161-d7da65079db7 None None] OVN-NB Sync routers and router ports started @ 2026-04-28 20:11:03.258241 {{(pid=106506) sync_routers_and_rports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:734}}
Apr 28 20:11:03.313754 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
Apr 28 20:11:03.314007 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d4918b1d-620c-42ce-a161-d7da65079db7 None None] OVN-NB Sync routers and router ports completed @ 2026-04-28 20:11:03.313824 {{(pid=106506) sync_routers_and_rports /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:942}}
Apr 28 20:11:03.314126 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d4918b1d-620c-42ce-a161-d7da65079db7 None None] OVN-NB Sync port QoS policies started @ 2026-04-28 20:11:03.313985 {{(pid=106506) sync_port_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1367}}
Apr 28 20:11:03.314224 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-d4918b1d-620c-42ce-a161-d7da65079db7 None None] Starting OVNClientQosExtension
Apr 28 20:11:03.330960 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d4918b1d-620c-42ce-a161-d7da65079db7 None None] OVN-NB Sync port QoS policies completed @ 2026-04-28 20:11:03.330529 {{(pid=106506) sync_port_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1390}}
Apr 28 20:11:03.330960 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d4918b1d-620c-42ce-a161-d7da65079db7 None None] OVN-NB Sync Floating IP QoS policies started @ 2026-04-28 20:11:03.330775 {{(pid=106506) sync_fip_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1395}}
Apr 28 20:11:03.331208 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [None req-d4918b1d-620c-42ce-a161-d7da65079db7 None None] Starting OVNClientQosExtension
Apr 28 20:11:03.356160 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d4918b1d-620c-42ce-a161-d7da65079db7 None None] OVN-NB Sync Floating IP QoS policies completed @ 2026-04-28 20:11:03.355895 {{(pid=106506) sync_fip_qos_policies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1418}}
Apr 28 20:11:03.356373 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d4918b1d-620c-42ce-a161-d7da65079db7 None None] OVN-NB Sync Floating IP NAT rules started @ 2026-04-28 20:11:03.356136 {{(pid=106506) sync_fip_dnat_rules /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1423}}
Apr 28 20:11:03.356533 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d4918b1d-620c-42ce-a161-d7da65079db7 None None] OVN-NB Sync Floating IP NAT rules completed @ 2026-04-28 20:11:03.356392 {{(pid=106506) sync_fip_dnat_rules /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1446}}
Apr 28 20:11:03.356634 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-d4918b1d-620c-42ce-a161-d7da65079db7 None None] OVN-Northbound DB sync process completed @ 2026-04-28 20:11:03.356517 {{(pid=106506) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:112}}
Apr 28 20:11:12.740175 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [-] OVN-Southbound DB sync process started @ 2026-04-28 20:11:12.739799 {{(pid=106506) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1474}}
Apr 28 20:11:12.741100 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
Apr 28 20:11:12.743148 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-05ebb763-a0f4-49e9-87bd-12c0918452bd None None] OVN-SB Sync hostname and physical networks started @ 2026-04-28 20:11:12.742673 {{(pid=106506) sync_hostname_and_physical_networks /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1501}}
Apr 28 20:11:12.745839 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.add_provnet_ext_id_to_localnet_ports {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.746014 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.746128 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_dhcp_options_consistency {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.746250 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_fair_meter_consistency {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.746364 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_fdb_aging_settings {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.746524 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_for_igmp_snoop_support {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.746664 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_for_inconsistencies {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.748382 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_global_dhcp_opts {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.748972 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.749075 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.749138 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_provider_distributed_ports {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.749259 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.749432 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.749509 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.cleanup_old_hash_ring_nodes {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.749597 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.configure_nb_global {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.749777 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.750191 np0000171185 neutron-ovn-maintenance-worker[106506]: WARNING neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-05ebb763-a0f4-49e9-87bd-12c0918452bd None None] New host np0000171185 found in OVN SB DB, but not in Neutron. Add its SegmentHostMapping in Neutron
Apr 28 20:11:12.750390 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_fip_distributed_flag {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.750577 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_network_type_and_physnet {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.751246 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.set_ovn_owned_dns_option {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.752641 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_ha_failover {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.752771 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.752837 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_mac_aging_settings {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.752897 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_router_distributed_flag {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.752984 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_router_static_routes {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.755950 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_security_group_with_address_group {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.756045 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [-] Periodic task found: DBInconsistenciesPeriodics.update_virtual_port_parent_hostname {{(pid=106506) add_periodics /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:148}}
Apr 28 20:11:12.757328 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.add_provnet_ext_id_to_localnet_ports' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.757729 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] Maintenance task thread has started
Apr 28 20:11:12.759074 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [-] MaintenanceWorker process has finished the post initialization
Apr 28 20:11:12.761915 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.762072 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_dhcp_options_consistency' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.762072 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fair_meter_consistency' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.762201 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.762276 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_igmp_snoop_support' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.762342 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.762403 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_global_dhcp_opts' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.762557 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.762557 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.762745 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_provider_distributed_ports' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.762745 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.762910 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.762910 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.cleanup_old_hash_ring_nodes' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.763044 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.configure_nb_global' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.763127 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.763127 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.763258 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_network_type_and_physnet' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.763330 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_ovn_owned_dns_option' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.763395 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.763479 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.763570 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.763570 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_distributed_flag' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.763825 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_static_routes' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.763825 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_security_group_with_address_group' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.763989 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting immediate callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_virtual_port_parent_hostname' {{(pid=106506) _process_immediates /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:826}}
Apr 28 20:11:12.770924 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG oslo_concurrency.lockutils [None req-05ebb763-a0f4-49e9-87bd-12c0918452bd None None] Acquiring lock "update_segment_host_mapping" by "neutron.services.segments.db.update_segment_host_mapping" {{(pid=106506) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:506}}
Apr 28 20:11:12.771231 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG oslo_concurrency.lockutils [None req-05ebb763-a0f4-49e9-87bd-12c0918452bd None None] Lock "update_segment_host_mapping" acquired by "neutron.services.segments.db.update_segment_host_mapping" :: waited 0.001s {{(pid=106506) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:519}}
Apr 28 20:11:12.795642 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron_lib.callbacks.manager [None req-05ebb763-a0f4-49e9-87bd-12c0918452bd None None] Publish callbacks ['neutron.plugins.ml2.plugin.DhcpAgentSchedulerDbMixin.auto_schedule_new_network_segments-2632243', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_host_addition_to_aggregate-1084284'] for segment_host_mapping (None), after_create {{(pid=106506) _notify_loop /opt/stack/data/venv/lib/python3.12/site-packages/neutron_lib/callbacks/manager.py:184}}
Apr 28 20:11:12.801531 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.add_provnet_ext_id_to_localnet_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106506) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 28 20:11:12.801768 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_baremetal_ports_dhcp_options {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:12.801901 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Sync baremetal port DHCP options with configuration. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:12.831709 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.services.segments.db [None req-05ebb763-a0f4-49e9-87bd-12c0918452bd None None] Segments {'7dbf713e-d963-4884-ae64-da4ce196e719'} mapped to the host np0000171185 {{(pid=106506) update_segment_host_mapping /opt/stack/neutron/neutron/services/segments/db.py:249}}
Apr 28 20:11:12.852466 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG oslo_concurrency.lockutils [None req-05ebb763-a0f4-49e9-87bd-12c0918452bd None None] Lock "update_segment_host_mapping" "released" by "neutron.services.segments.db.update_segment_host_mapping" :: held 0.081s {{(pid=106506) inner /opt/stack/data/venv/lib/python3.12/site-packages/oslo_concurrency/lockutils.py:538}}
Apr 28 20:11:12.852640 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-05ebb763-a0f4-49e9-87bd-12c0918452bd None None] OVN-SB Sync hostname and physical networks completed @ 2026-04-28 20:11:12.852463 {{(pid=106506) sync_hostname_and_physical_networks /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1528}}
Apr 28 20:11:12.861434 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_baremetal_ports_dhcp_options finished in 0.059 seconds
Apr 28 20:11:12.861741 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_baremetal_ports_dhcp_options' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106506) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 28 20:11:12.861929 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_dhcp_options_consistency {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:12.862127 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Backfill network_id on DHCP_Options external_ids. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:12.919849 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_dhcp_options_consistency finished in 0.057 seconds
Apr 28 20:11:12.920037 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_dhcp_options_consistency' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106506) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 28 20:11:12.920235 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fair_meter_consistency {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:12.920360 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Refresh ACL logging fair meter after configuration reload. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:12.920670 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fair_meter_consistency finished in 0.000 seconds
Apr 28 20:11:12.920821 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fair_meter_consistency' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106506) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 28 20:11:12.920971 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:12.921091 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:12.923475 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:13.028722 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:13.029206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.031754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:13.031754 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:13.031754 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:13.031754 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:13.031754 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:13.031754 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:13.031754 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.031754 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:13.047380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.111 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.047380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:13.047380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:13.047380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:13.047380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.047380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:11:13.047380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:13.047380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.047380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:13.047380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:13.047380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:13.047380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.048156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:13.048156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:13.048156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:13.048156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:13.048156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:13.048156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:13.048156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:13.048156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:13.048156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:13.048156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:13.048156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:13.048156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:13.048156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:13.048856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.048856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:13.051991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.051991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:13.051991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:13.051991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:13.051991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:13.051991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:13.051991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.051991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:13.051991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:13.051991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.051991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:13.051991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:13.051991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.051991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:13.051991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:13.051991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.052914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:11:13.052914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:13.052914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.052914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:13.052914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:13.052914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:13.052914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.052914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:13.052914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:13.052914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:13.052914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:13.052914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:13.052914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:13.052914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:13.052914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:13.052914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:13.052914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:13.052914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:13.052914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:13.054253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:13.054253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.054253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:13.054253 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_for_igmp_snoop_support {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:13.054253 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Check for IGMP snoop support. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:13.054253 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_for_igmp_snoop_support finished in 0.000 seconds
Apr 28 20:11:13.054253 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_igmp_snoop_support' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106506) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 28 20:11:13.054253 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:13.054985 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:13.084258 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=ed654c20-61c6-4e90-a5d3-648d4fbccbf3, enabled=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-7ff08466-af6d-4e13-bb1d-8987170a5734', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-15fdd774-e739-4763-a086-bc4b40fb7431', 'neutron:subnet_ids': '4b461bd9-09ea-4f87-a007-a11ab022ff7f'}, gateway_chassis=[], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={'address_mode': 'slaac', 'mtu': '1372', 'send_periodic': 'true'}, mac='fa:16:3e:82:5e:06', name='lrp-35dccb9f-6f17-4fdc-94ff-62d86f803629', networks=['fd33:b013:f9ec::1/64'], options={}, peer=[], status={}) old= {{(pid=106506) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
Apr 28 20:11:13.092090 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.extensions.qos [-] Starting OVNClientQosExtension
Apr 28 20:11:13.144751 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: No inconsistencies found. Skipping {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:437}}
Apr 28 20:11:13.145240 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_for_inconsistencies finished in 0.092 seconds
Apr 28 20:11:13.145592 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_global_dhcp_opts {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:13.145797 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Check global DHCP options consistency. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:13.146067 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_global_dhcp_opts finished in 0.000 seconds
Apr 28 20:11:13.146242 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_global_dhcp_opts' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106506) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 28 20:11:13.146413 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_localnet_port_has_learn_fdb {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:13.146517 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Ensure localnet ports have learn-fdb set per configuration. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:13.157521 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
Apr 28 20:11:13.157862 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_localnet_port_has_learn_fdb finished in 0.011 seconds
Apr 28 20:11:13.158013 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_localnet_port_has_learn_fdb' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106506) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 28 20:11:13.158201 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_network_broadcast_arps_to_all_routers {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:13.158317 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Sync broadcast-arps-to-all-routers on external networks. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:13.280456 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-7ff08466-af6d-4e13-bb1d-8987170a5734 synchronized; highest priority chassis d6012b70-0a69-43bb-a268-b8c8213be6d4
Apr 28 20:11:13.282461 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-7ff08466-af6d-4e13-bb1d-8987170a5734, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': '7ff08466-af6d-4e13-bb1d-8987170a5734', 'neutron:router_id': '15fdd774-e739-4763-a086-bc4b40fb7431'}),), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:13.282750 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-7ff08466-af6d-4e13-bb1d-8987170a5734, chassis=d6012b70-0a69-43bb-a268-b8c8213be6d4, priority=1, columns={}) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:13.282951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.283815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 28 20:11:13.283815 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:13.283815 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:13.283815 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:13.283815 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:13.283815 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:13.283815 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.283815 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:13.295064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.295064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event Traceback (most recent call last):
Apr 28 20:11:13.295064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
Apr 28 20:11:13.295064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     match.run(event, row, updates)
Apr 28 20:11:13.295064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 86, in run
Apr 28 20:11:13.295064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.link_network_ha_chassis_group(
Apr 28 20:11:13.295064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2411, in link_network_ha_chassis_group
Apr 28 20:11:13.295064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:13.295064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.295064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     next(self.gen)
Apr 28 20:11:13.295064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:13.295064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:13.295064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.295064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     next(self.gen)
Apr 28 20:11:13.295064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:13.295064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     with self.create_transaction(
Apr 28 20:11:13.295064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:13.295064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     self.result = self.commit()
Apr 28 20:11:13.295064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
Apr 28 20:11:13.295827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:13.295827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     raise result.ex
Apr 28 20:11:13.295827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:13.295827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
Apr 28 20:11:13.295827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
Apr 28 20:11:13.295827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:13.295827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     raise RuntimeError(msg)
Apr 28 20:11:13.295827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.295827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event 
Apr 28 20:11:13.303032 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_network_broadcast_arps_to_all_routers finished in 0.144 seconds
Apr 28 20:11:13.303151 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_network_broadcast_arps_to_all_routers' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106506) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 28 20:11:13.303331 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_provider_distributed_ports {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:13.303441 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Align reside-on-redirect-chassis on VLAN/FLAT distributed ports. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:13.401365 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_provider_distributed_ports finished in 0.098 seconds
Apr 28 20:11:13.402482 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_provider_distributed_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106506) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 28 20:11:13.402801 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_redirect_type_router_gateway_ports {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:13.402801 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Align router gateway redirect-type for provider VLAN/FLAT networks. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:13.650695 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
Apr 28 20:11:13.652301 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_db_sync [None req-05ebb763-a0f4-49e9-87bd-12c0918452bd None None] OVN-Southbound DB sync process completed @ 2026-04-28 20:11:13.651761 {{(pid=106506) do_sync /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_db_sync.py:1482}}
Apr 28 20:11:13.699936 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_redirect_type_router_gateway_ports finished in 0.297 seconds
Apr 28 20:11:13.700112 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_redirect_type_router_gateway_ports' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106506) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 28 20:11:13.700502 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:13.700502 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:13.702033 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:13.702404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.703388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:13.703388 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:13.703388 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:13.703388 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:13.703388 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:13.703388 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:13.703388 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.703388 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:13.704384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.704384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:13.704384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:13.704384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:13.704384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.704384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:11:13.704384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:13.704384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.704384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:13.704384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:13.704384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:13.704384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.704912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:13.704912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:13.704912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:13.704912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:13.704912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:13.704912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:13.704912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:13.704912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:13.704912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:13.704912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:13.704912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:13.704912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:13.704912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:13.705230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.705230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:13.705370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.705370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:13.705370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:13.705370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:13.705370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:13.705370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:13.705370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.705370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:13.705370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:13.705370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.705370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:13.705370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:13.705370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.705370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:13.705370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:13.705370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.705796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:11:13.705796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:13.705796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.705796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:13.705796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:13.705796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:13.705796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.705796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:13.705796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:13.705796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:13.705796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:13.705796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:13.705796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:13.705796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:13.705796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:13.705796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:13.705796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:13.705796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:13.705796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:13.706263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:13.706263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.706263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:13.706263 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: cleanup_old_hash_ring_nodes {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:13.706263 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Purge stale hash ring nodes older than five days. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:13.711730 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.db.ovn_hash_ring_db [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Cleaned up Hash Ring nodes older than 5 days
Apr 28 20:11:13.711881 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task cleanup_old_hash_ring_nodes finished in 0.006 seconds
Apr 28 20:11:13.712291 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: configure_nb_global {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:13.712395 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply ovn_nb_global settings to NB_Global options. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:13.712753 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'ignore_lsp_down': 'false', 'fdb_removal_limit': '0', 'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:13.713435 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'ignore_lsp_down': 'false', 'fdb_removal_limit': '0', 'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:13.714048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.715002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:13.715002 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:13.715002 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:13.715002 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:13.715002 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:13.715002 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:13.715002 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.715002 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:13.715973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.715973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:11:13.715973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:13.715973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:13.715973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.715973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:13.715973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:13.715973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:13.715973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.715973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:13.715973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:13.715973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:11:13.715973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:13.715973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:11:13.715973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:11:13.716876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:13.716876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:11:13.716876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:13.716876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:11:13.716876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:11:13.716876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:13.716876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:11:13.716876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.716876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:11:13.717359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task configure_nb_global failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.717359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:13.717359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:13.717359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:13.717359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.717359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1038, in configure_nb_global
Apr 28 20:11:13.717359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:11:13.717359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:13.717359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:13.717359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.717359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:13.717922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:13.717922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:13.717922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.717922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:13.717922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:13.717922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:13.717922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:13.717922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:13.717922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:13.717922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:13.717922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:13.717922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:13.717922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:13.717922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:13.718528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:13.718528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:13.718528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.718528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:13.718528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.configure_nb_global' (it runs every 86400.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.718528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:13.718528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:13.718528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:13.718528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:13.718528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:13.718528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.718528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:13.718528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:13.718528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.719220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:13.719220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:13.719220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.719220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:13.719220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:13.719220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.719220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1038, in configure_nb_global
Apr 28 20:11:13.719220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:11:13.719220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:13.719220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:13.719220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.719220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:13.719220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:13.719220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:13.719220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.719220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:13.719220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:13.719220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:13.719220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:13.720571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:13.720571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:13.720571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:13.720571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:13.720571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:13.720571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:13.720571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:13.720571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:13.720571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:13.720571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.720571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:13.720571 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] There are no commands to commit {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:78}}
Apr 28 20:11:13.720571 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.migrate_lrp_gateway_chassis_to_ha_chassis_group' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106506) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 28 20:11:13.721034 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:13.721034 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:13.721034 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:11:13.721034 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:13.721034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.721320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:13.721320 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:13.721320 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:13.721320 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:13.721320 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:13.721320 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:13.721320 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.721320 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:13.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:11:13.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:13.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:13.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:13.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:13.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:13.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:13.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:13.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:11:13.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:13.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:11:13.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:11:13.722759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:13.722759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:11:13.722759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:13.722759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:11:13.722759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:11:13.722759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:13.722759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:11:13.722759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.722759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:11:13.722759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.722759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:13.722759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:13.722759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:13.722759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.723187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:11:13.723187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:11:13.723187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:11:13.723187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:13.723187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:13.723187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.723187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:13.723187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:13.723187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:13.723187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.723187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:13.723187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:13.723187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:13.723606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:13.723606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:13.723606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:13.723606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:13.723606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:13.723606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:13.723606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:13.723606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:13.723606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:13.723606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:13.723606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.723606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:13.724218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.724218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:13.724218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:13.724218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:13.724218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:13.724218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:13.724218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.724218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:13.724218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:13.724218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.724218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:13.724218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:13.724218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.724218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:13.724218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:13.724218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.724758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:11:13.724758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:11:13.724758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:11:13.724758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:13.724758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:13.724758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.724758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:13.724758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:13.724758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:13.724758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.724758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:13.724758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:13.724758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:13.724758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:13.724758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:13.724758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:13.724758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:13.724758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:13.725296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:13.725296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:13.725296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:13.725296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:13.725296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:13.725296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.725296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:13.725296 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_network_type_and_physnet {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:13.725296 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set network type and physnet external_ids on logical switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:13.734375 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_network_type_and_physnet finished in 0.010 seconds
Apr 28 20:11:13.734485 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_network_type_and_physnet' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106506) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 28 20:11:13.734536 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_ovn_owned_dns_option {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:13.734632 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply ovn_owned option on DNS records per configuration. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:13.734917 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_ovn_owned_dns_option finished in 0.000 seconds
Apr 28 20:11:13.734978 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_ovn_owned_dns_option' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106506) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 28 20:11:13.735073 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:13.735298 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:13.735543 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:13.736041 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:13.736347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.737038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:13.737038 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:13.737038 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:13.737038 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:13.737038 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:13.737038 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:13.737038 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.737038 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:13.738019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.738019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:11:13.738019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:13.738019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:13.738019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.738019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:13.738019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:13.738019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:13.738019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.738019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:13.738019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:13.738019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:11:13.738019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:13.738019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:11:13.738019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:11:13.738510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:13.738510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:11:13.738510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:13.738510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:11:13.738510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:11:13.738510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:13.738510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:11:13.738510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.738510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:11:13.738804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.738804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:13.738804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:13.738804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:13.738804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.738804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:11:13.738804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:11:13.738804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:13.738804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:13.738804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.738804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:13.739158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:13.739158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:13.739158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.739158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:13.739158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:13.739158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:13.739158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:13.739158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:13.739158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:13.739158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:13.739158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:13.739158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:13.739158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:13.739158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:13.739569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:13.739569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:13.739569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.739569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:13.739569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.739569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:13.739569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:13.739569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:13.739569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:13.739569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:13.739569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.739569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:13.739569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:13.739569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.740018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:13.740018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:13.740018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.740018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:13.740018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:13.740018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.740018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:11:13.740018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:11:13.740018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:13.740018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:13.740018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.740018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:13.740018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:13.740018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:13.740018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.740018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:13.740018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:13.740018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:13.740018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:13.741001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:13.741001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:13.741001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:13.741001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:13.741001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:13.741001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:13.741001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:13.741001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:13.741001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:13.741001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.741001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:13.741001 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_lrouter_ports_ext_ids_name_prefix {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:13.741001 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: ('Check missing prefix router_name', 'in external_ids of LRPs.') {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:13.741407 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_lrouter_ports_ext_ids_name_prefix finished in 0.001 seconds
Apr 28 20:11:13.741407 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_lrouter_ports_ext_ids_name_prefix' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106506) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 28 20:11:13.741407 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:13.741407 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:13.741407 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:13.741806 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:13.742199 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:13.742457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.743285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:13.743285 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:13.743285 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:13.743285 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:13.743285 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:13.743285 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:13.743285 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.743285 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:13.743958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.743958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:13.743958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:13.743958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:13.743958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.743958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:11:13.743958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:13.743958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.743958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:13.743958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:13.743958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:13.743958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.744502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:13.744502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:13.744502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:13.744502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:13.744502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:13.744502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:13.744502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:13.744502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:13.744502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:13.744502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:13.744502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:13.744502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:13.744502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:13.744939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.744939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:13.744939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call immediate 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.744939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:13.744939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:13.744939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:13.744939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:13.744939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:13.744939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.744939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:13.744939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:13.744939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.744939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:13.744939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:13.744939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.745703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:13.745703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:13.745703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:13.745703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:11:13.745703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:13.745703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.745703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:13.745703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:13.745703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:13.745703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:13.745703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:13.745703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:13.745703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:13.745703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:13.745703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:13.745703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:13.745703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:13.745703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:13.745703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:13.746547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:13.746547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:13.746547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:13.746547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:13.746547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:13.746547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:13.746547 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_router_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:13.746547 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Sync router.distributed with distributed floating IP config. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:13.748125 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_router_distributed_flag finished in 0.003 seconds
Apr 28 20:11:13.748426 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_distributed_flag' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106506) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 28 20:11:13.748787 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_router_static_routes {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:13.749027 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Mark Neutron-owned static routes in OVN external_ids. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:13.749703 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_router_static_routes finished in 0.001 seconds
Apr 28 20:11:13.749851 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_router_static_routes' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106506) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 28 20:11:13.750120 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_security_group_with_address_group {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:13.750371 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Repair Address_Sets and ACLs for address group rules. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:13.764955 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_security_group_with_address_group finished in 0.014 seconds
Apr 28 20:11:13.765094 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_security_group_with_address_group' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106506) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 28 20:11:13.766231 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_virtual_port_parent_hostname' raised 'NeverAgain' exception, stopping any further execution of it. {{(pid=106506) _on_done /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:881}}
Apr 28 20:11:15.017988 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=4eaa171a-b115-4a74-94b3-d22458be32fc, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-8a1fb774-5693-4139-8b75-6ab5273228c7', 'neutron:revision_number': '8', 'neutron:router_name': 'neutron-15fdd774-e739-4763-a086-bc4b40fb7431', 'neutron:subnet_ids': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece 335d12ce-2d0c-4086-b3f4-2af390c03d70'}, gateway_chassis=[], ha_chassis_group=f24fa228-2c7b-42cf-a99f-0fb59c95cd43, ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:44:eb:86', name='lrp-f7ffb86e-b548-4e06-99b1-8dd93536a362', networks=['172.24.5.85/24', '2001:db8::1/64'], options={'gateway_mtu': '1372', 'reside-on-redirect-chassis': 'true'}, peer=[], status={}) old= {{(pid=106506) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
Apr 28 20:11:15.019385 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=cea86df0-883c-46be-9ccb-1cd2123d042c, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-8a1fb774-5693-4139-8b75-6ab5273228c7', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-15fdd774-e739-4763-a086-bc4b40fb7431', 'neutron:subnet_ids': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece'}, gateway_chassis=[], ha_chassis_group=f24fa228-2c7b-42cf-a99f-0fb59c95cd43, ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:44:eb:86', name='lrp-f7ffb86e-b548-4e06-99b1-8dd93536a362', networks=['172.24.5.85/24'], options={'gateway_mtu': '1372', 'reside-on-redirect-chassis': 'true'}, peer=[], status={'hosting-chassis': 'd6012b70-0a69-43bb-a268-b8c8213be6d4'}) old= {{(pid=106506) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
Apr 28 20:11:15.067359 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-7ff08466-af6d-4e13-bb1d-8987170a5734 synchronized; highest priority chassis d6012b70-0a69-43bb-a268-b8c8213be6d4
Apr 28 20:11:15.068191 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-7ff08466-af6d-4e13-bb1d-8987170a5734, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': '7ff08466-af6d-4e13-bb1d-8987170a5734', 'neutron:router_id': '15fdd774-e739-4763-a086-bc4b40fb7431'}),), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:15.068761 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-7ff08466-af6d-4e13-bb1d-8987170a5734, chassis=d6012b70-0a69-43bb-a268-b8c8213be6d4, priority=1, columns={}) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:15.069227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:15.070043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 28 20:11:15.070043 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:15.070043 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:15.070043 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:15.070043 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:15.070043 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:15.070043 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:15.070043 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:15.070954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:15.070954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event Traceback (most recent call last):
Apr 28 20:11:15.070954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
Apr 28 20:11:15.070954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     match.run(event, row, updates)
Apr 28 20:11:15.070954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 89, in run
Apr 28 20:11:15.070954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.update_router_ha_chassis_group(
Apr 28 20:11:15.070954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1664, in update_router_ha_chassis_group
Apr 28 20:11:15.070954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     self.link_network_ha_chassis_group(context, network_id, router_id)
Apr 28 20:11:15.070954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2411, in link_network_ha_chassis_group
Apr 28 20:11:15.070954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:15.070954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:15.070954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     next(self.gen)
Apr 28 20:11:15.070954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:15.070954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:15.070954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:15.070954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     next(self.gen)
Apr 28 20:11:15.070954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:15.070954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     with self.create_transaction(
Apr 28 20:11:15.071544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:15.071544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     self.result = self.commit()
Apr 28 20:11:15.071544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
Apr 28 20:11:15.071544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:15.071544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     raise result.ex
Apr 28 20:11:15.071544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:15.071544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
Apr 28 20:11:15.071544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
Apr 28 20:11:15.071544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:15.071544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     raise RuntimeError(msg)
Apr 28 20:11:15.071544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:15.071544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event 
Apr 28 20:11:15.111554 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-7ff08466-af6d-4e13-bb1d-8987170a5734 synchronized; highest priority chassis d6012b70-0a69-43bb-a268-b8c8213be6d4
Apr 28 20:11:15.112062 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=HA_Chassis_Group, record=neutron-7ff08466-af6d-4e13-bb1d-8987170a5734, col_values=(('external_ids', {'neutron:availability_zone_hints': '', 'neutron:network_id': '7ff08466-af6d-4e13-bb1d-8987170a5734', 'neutron:router_id': '15fdd774-e739-4763-a086-bc4b40fb7431'}),), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:15.112318 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-7ff08466-af6d-4e13-bb1d-8987170a5734, chassis=d6012b70-0a69-43bb-a268-b8c8213be6d4, priority=1, columns={}) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:15.112488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:15.113126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 28 20:11:15.113126 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:15.113126 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:15.113126 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:15.113126 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:15.113126 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:15.113126 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:15.113126 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:15.114546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:15.114546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event Traceback (most recent call last):
Apr 28 20:11:15.114546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
Apr 28 20:11:15.114546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     match.run(event, row, updates)
Apr 28 20:11:15.114546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 81, in run
Apr 28 20:11:15.114546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.update_router_ha_chassis_group(
Apr 28 20:11:15.114546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1664, in update_router_ha_chassis_group
Apr 28 20:11:15.114546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     self.link_network_ha_chassis_group(context, network_id, router_id)
Apr 28 20:11:15.114546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2411, in link_network_ha_chassis_group
Apr 28 20:11:15.114546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:15.114546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:15.114546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     next(self.gen)
Apr 28 20:11:15.114546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:15.114546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:15.114546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:15.114546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     next(self.gen)
Apr 28 20:11:15.114546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:15.114546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     with self.create_transaction(
Apr 28 20:11:15.115106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:15.115106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     self.result = self.commit()
Apr 28 20:11:15.115106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
Apr 28 20:11:15.115106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:15.115106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     raise result.ex
Apr 28 20:11:15.115106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:15.115106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
Apr 28 20:11:15.115106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
Apr 28 20:11:15.115106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:15.115106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     raise RuntimeError(msg)
Apr 28 20:11:15.115106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:15.115106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event 
Apr 28 20:11:17.921451 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:17.922136 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:17.922230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:17.923622 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:17.957936 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:17.958175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:17.958818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:17.958818 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:17.958818 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:17.958818 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:17.958818 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:17.958818 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:17.958818 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:17.958818 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:17.959745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:17.959745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:17.959745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:17.959745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:17.959745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:17.959745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:11:17.959745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:17.959745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:17.959745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:17.959745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:17.959745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:17.959745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:17.960147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:17.960147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:17.960147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:17.960147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:17.960147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:17.960147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:17.960147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:17.960147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:17.960147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:17.960147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:17.960147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:17.960147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:17.960147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:17.960499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:17.960499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:17.960626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:17.960626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:17.960626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:17.960626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:17.960626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:17.960626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:17.960626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:17.960626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:17.960626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:17.960626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:17.960626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:17.960626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:17.960626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:17.960626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:17.960626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:17.960626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:17.961112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:11:17.961112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:17.961112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:17.961112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:17.961112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:17.961112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:17.961112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:17.961112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:17.961112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:17.961112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:17.961112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:17.961112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:17.961112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:17.961112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:17.961112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:17.961112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:17.961112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:17.961112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:17.961112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:17.961646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:17.961646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:17.961646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:18.700522 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:18.701178 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:18.701440 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:18.702407 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:18.702704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.703625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:18.703625 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:18.703625 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:18.703625 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:18.703625 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:18.703625 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:18.703625 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.703625 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:18.705305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.705305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:18.705305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:18.705305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:18.705305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:18.705305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:11:18.705305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:18.705305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:18.705305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:18.705305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:18.705305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:18.705305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:18.706113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:18.706113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:18.706113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:18.706113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:18.706113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:18.706113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:18.706113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:18.706113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:18.706113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:18.706113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:18.706113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:18.706113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:18.706113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:18.707081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.707081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:18.707821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.707821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:18.707821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:18.707821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:18.707821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:18.707821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:18.707821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:18.707821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:18.707821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:18.707821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:18.707821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:18.707821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:18.707821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:18.707821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:18.707821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:18.707821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:18.708330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:11:18.708330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:18.708330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:18.708330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:18.708330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:18.708330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:18.708330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:18.708330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:18.708330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:18.708330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:18.708330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:18.708330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:18.708330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:18.708330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:18.708330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:18.708330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:18.708330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:18.708330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:18.708330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:18.708857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:18.708857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.708857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:18.720746 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:18.721150 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:18.721331 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:18.721516 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:11:18.722057 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:18.722380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.723267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:18.723267 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:18.723267 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:18.723267 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:18.723267 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:18.723267 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:18.723267 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.723267 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:18.724491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.724491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:11:18.724491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:18.724491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:18.724491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:18.724491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:18.724491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:18.724491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:18.724491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:18.724491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:18.724491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:18.724491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:11:18.724491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:18.724491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:11:18.724491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:11:18.725303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:18.725303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:11:18.725303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:18.725303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:11:18.725303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:11:18.725303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:18.725303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:11:18.725303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.725303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:11:18.726227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.726227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:18.726227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:18.726227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:18.726227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:18.726227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:11:18.726227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:11:18.726227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:11:18.726227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:18.726227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:18.726227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:18.726227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:18.727018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:18.727018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:18.727018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:18.727018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:18.727018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:18.727018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:18.727018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:18.727018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:18.727018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:18.727018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:18.727018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:18.727018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:18.727018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:18.727018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:18.727722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:18.727722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:18.727722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.727722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:18.728261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.728261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:18.728261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:18.728261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:18.728261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:18.728261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:18.728261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:18.728261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:18.728261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:18.728261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:18.728261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:18.728261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:18.728261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:18.728261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:18.728261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:18.728261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:18.729207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:11:18.729207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:11:18.729207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:11:18.729207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:18.729207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:18.729207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:18.729207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:18.729207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:18.729207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:18.729207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:18.729207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:18.729207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:18.729207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:18.729207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:18.729207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:18.729207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:18.729207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:18.729207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:18.730183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:18.730183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:18.730183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:18.730183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:18.730183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:18.730183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.730183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:18.735305 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:18.735586 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:18.735724 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:18.735901 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:18.736259 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:18.736531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.737184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:18.737184 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:18.737184 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:18.737184 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:18.737184 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:18.737184 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:18.737184 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.737184 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:18.738541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.738541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:11:18.738541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:18.738541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:18.738541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:18.738541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:18.738541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:18.738541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:18.738541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:18.738541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:18.738541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:18.738541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:11:18.738541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:18.738541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:11:18.738541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:11:18.738984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:18.738984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:11:18.738984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:18.738984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:11:18.738984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:11:18.738984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:18.738984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:11:18.738984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.738984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:11:18.740244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.740244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:18.740244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:18.740244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:18.740244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:18.740244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:11:18.740244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:11:18.740244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:18.740244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:18.740244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:18.740244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:18.740962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:18.740962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:18.740962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:18.740962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:18.740962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:18.740962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:18.740962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:18.740962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:18.740962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:18.740962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:18.740962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:18.740962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:18.740962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:18.740962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:18.741766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:18.741766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:18.741766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.741766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:18.742926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.742926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:18.742926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:18.742926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:18.742926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:18.742926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:18.742926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:18.742926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:18.742926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:18.742926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:18.742926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:18.742926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:18.742926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:18.742926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:18.742926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:18.742926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:18.742926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:11:18.743865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:11:18.743865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:18.743865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:18.743865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:18.743865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:18.743865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:18.743865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:18.743865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:18.743865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:18.743865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:18.743865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:18.743865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:18.743865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:18.743865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:18.743865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:18.743865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:18.743865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:18.743865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:18.743865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:18.744771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:18.744771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:18.744771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.744771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:18.744771 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:18.744771 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:18.744771 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:18.744771 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:18.745277 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:18.745277 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:18.745277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.745482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:18.745482 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:18.745482 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:18.745482 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:18.745482 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:18.745482 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:18.745482 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.745482 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:18.746053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.746053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:18.746053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:18.746053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:18.746053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:18.746053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:11:18.746053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:18.746053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:18.746053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:18.746053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:18.746053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:18.746053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:18.746723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:18.746723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:18.746723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:18.746723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:18.746723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:18.746723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:18.746723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:18.746723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:18.746723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:18.746723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:18.746723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:18.746723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:18.746723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:18.747377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.747377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:18.747377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.747377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:18.747377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:18.747377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:18.747377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:18.747377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:18.747377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:18.747377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:18.747377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:18.747377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:18.747377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:18.747377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:18.747377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:18.748063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:18.748063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:18.748063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:18.748063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:11:18.748063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:18.748063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:18.748063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:18.748063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:18.748063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:18.748063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:18.748063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:18.748063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:18.748063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:18.748063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:18.748063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:18.748063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:18.748063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:18.748063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:18.748063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:18.748569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:18.748569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:18.748569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:18.748569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:18.748569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:18.748569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:22.922317 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:22.923072 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:22.923072 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:22.924180 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:22.962546 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:22.962860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:22.963581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:22.963581 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:22.963581 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:22.963581 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:22.963581 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:22.963581 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:22.963581 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:22.963581 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:22.964773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:22.964773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:22.964773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:22.964773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:22.964773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:22.964773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:11:22.964773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:22.964773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:22.964773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:22.964773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:22.964773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:22.964773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:22.965491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:22.965491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:22.965491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:22.965491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:22.965491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:22.965491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:22.965491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:22.965491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:22.965491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:22.965491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:22.965491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:22.965491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:22.965491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:22.966104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:22.966104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:22.966104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:22.966104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:22.966104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:22.966104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:22.966104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:22.966104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:22.966104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:22.966104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:22.966104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:22.966104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:22.966104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:22.966104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:22.966104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:22.966877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:22.966877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:22.966877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:22.966877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:11:22.966877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:22.966877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:22.966877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:22.966877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:22.966877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:22.966877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:22.966877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:22.966877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:22.966877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:22.966877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:22.966877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:22.966877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:22.966877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:22.966877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:22.966877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:22.967894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:22.967894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:22.967894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:22.967894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:22.967894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:22.967894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:23.701077 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:23.702359 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:23.702816 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:23.703598 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:23.703904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.704603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:23.704603 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:23.704603 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:23.704603 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:23.704603 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:23.704603 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:23.704603 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.704603 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:23.705757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.705757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:23.705757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:23.705757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:23.705757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:23.705757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:11:23.705757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:23.705757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:23.705757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:23.705757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:23.705757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:23.705757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:23.706353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:23.706353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:23.706353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:23.706353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:23.706353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:23.706353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:23.706353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:23.706353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:23.706353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:23.706353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:23.706353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:23.706353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:23.706353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:23.706934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.706934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:23.706934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.706934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:23.706934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:23.706934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:23.706934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:23.706934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:23.706934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:23.706934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:23.706934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:23.706934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:23.706934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:23.706934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:23.706934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:23.707658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:23.707658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:23.707658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:23.707658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:11:23.707658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:23.707658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:23.707658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:23.707658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:23.707658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:23.707658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:23.707658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:23.707658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:23.707658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:23.707658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:23.707658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:23.707658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:23.707658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:23.707658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:23.707658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:23.708593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:23.708593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:23.708593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:23.708593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:23.708593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.708593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:23.721311 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:23.721848 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:23.722067 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:23.722273 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:11:23.722976 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:23.723376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.724431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:23.724431 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:23.724431 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:23.724431 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:23.724431 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:23.724431 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:23.724431 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.724431 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:23.726119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.726119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:11:23.726119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:23.726119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:23.726119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:23.726119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:23.726119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:23.726119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:23.726119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:23.726119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:23.726119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:23.726119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:11:23.726119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:23.726119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:11:23.726119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:11:23.726881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:23.726881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:11:23.726881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:23.726881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:11:23.726881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:11:23.726881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:23.726881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:11:23.726881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.726881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:11:23.727269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.727269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:23.727269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:23.727269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:23.727269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:23.727269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:11:23.727269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:11:23.727269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:11:23.727269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:23.727269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:23.727269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:23.727269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:23.727961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:23.727961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:23.727961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:23.727961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:23.727961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:23.727961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:23.727961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:23.727961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:23.727961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:23.727961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:23.727961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:23.727961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:23.727961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:23.727961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:23.728649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:23.728649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:23.728649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.728649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:23.728649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.728649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:23.728649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:23.728649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:23.728649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:23.728649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:23.728649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:23.728649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:23.728649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:23.728649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:23.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:23.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:23.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:23.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:23.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:23.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:23.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:11:23.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:11:23.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:11:23.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:23.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:23.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:23.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:23.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:23.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:23.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:23.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:23.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:23.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:23.730368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:23.730368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:23.730368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:23.730368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:23.730368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:23.730368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:23.730368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:23.730368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:23.730368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:23.730368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:23.730368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.730368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:23.735826 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:23.736235 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:23.736448 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:23.736783 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:23.737526 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:23.738041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.738967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:23.738967 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:23.738967 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:23.738967 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:23.738967 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:23.738967 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:23.738967 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.738967 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:23.739856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.739856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:11:23.739856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:23.739856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:23.739856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:23.739856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:23.739856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:23.739856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:23.739856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:23.739856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:23.739856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:23.739856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:11:23.739856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:23.739856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:11:23.739856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:11:23.740862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:23.740862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:11:23.740862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:23.740862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:11:23.740862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:11:23.740862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:23.740862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:11:23.740862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.740862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:11:23.740862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.740862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:23.740862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:23.740862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:23.740862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:23.741733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:11:23.741733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:11:23.741733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:23.741733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:23.741733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:23.741733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:23.741733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:23.741733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:23.741733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:23.741733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:23.741733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:23.741733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:23.741733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:23.742272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:23.742272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:23.742272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:23.742272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:23.742272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:23.742272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:23.742272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:23.742272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:23.742272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:23.742272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.742272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:23.742272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.742618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:23.742618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:23.742618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:23.742618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:23.742618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:23.742618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:23.742618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:23.742618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:23.742618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:23.742618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:23.742618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:23.742618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:23.742618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:23.742618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:23.742618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:23.742618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:11:23.742618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:11:23.742618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:23.742618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:23.743320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:23.743320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:23.743320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:23.743320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:23.743320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:23.743320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:23.743320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:23.743320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:23.743320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:23.743320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:23.743320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:23.743320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:23.743320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:23.743320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:23.743320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:23.743320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:23.743320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:23.743320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:23.743320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.743320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:23.744091 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:23.744091 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:23.744091 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:23.744091 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:23.744593 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:23.744998 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:23.745189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.746029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:23.746029 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:23.746029 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:23.746029 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:23.746029 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:23.746029 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:23.746029 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.746029 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:23.746947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.746947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:23.746947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:23.746947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:23.746947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:23.746947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:11:23.746947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:23.746947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:23.746947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:23.746947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:23.746947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:23.746947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:23.747309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:23.747309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:23.747309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:23.747309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:23.747309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:23.747309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:23.747309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:23.747309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:23.747309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:23.747309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:23.747309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:23.747309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:23.747309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:23.747657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.747657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:23.747965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.747965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:23.747965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:23.747965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:23.747965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:23.747965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:23.747965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:23.747965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:23.747965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:23.747965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:23.747965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:23.747965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:23.747965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:23.747965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:23.747965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:23.747965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:23.748388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:11:23.748388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:23.748388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:23.748388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:23.748388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:23.748388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:23.748388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:23.748388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:23.748388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:23.748388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:23.748388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:23.748388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:23.748388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:23.748388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:23.748388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:23.748388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:23.748388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:23.748388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:23.748388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:23.748893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:23.748893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:23.748893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:27.922888 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:27.923958 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:27.923958 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:27.925308 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:27.964049 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:27.964380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:27.965371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:27.965371 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:27.965371 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:27.965371 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:27.965371 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:27.965371 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:27.965371 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:27.965371 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:27.966154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:27.966154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:27.966154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:27.966154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:27.966154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:27.966154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:11:27.966154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:27.966154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:27.966154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:27.966154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:27.966154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:27.966154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:27.967114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:27.967114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:27.967114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:27.967114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:27.967114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:27.967114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:27.967114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:27.967114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:27.967114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:27.967114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:27.967114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:27.967114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:27.967114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:27.968044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:27.968044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:27.968044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:27.968044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:27.968044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:27.968044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:27.968044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:27.968044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:27.968044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:27.968044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:27.968044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:27.968044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:27.968044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:27.968044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:27.968044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:27.968672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:27.968672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:27.968672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:27.968672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:11:27.968672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:27.968672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:27.968672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:27.968672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:27.968672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:27.968672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:27.968672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:27.968672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:27.968672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:27.968672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:27.968672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:27.968672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:27.968672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:27.968672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:27.968672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:27.969514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:27.969514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:27.969514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:27.969514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:27.969514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:27.969514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:28.701653 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:28.702283 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:28.702476 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:28.703476 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:28.703927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.705073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:28.705073 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:28.705073 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:28.705073 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:28.705073 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:28.705073 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:28.705073 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.705073 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:28.706509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.706509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:28.706509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:28.706509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:28.706509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:28.706509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:11:28.706509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:28.706509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:28.706509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:28.706509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:28.706509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:28.706509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:28.707342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:28.707342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:28.707342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:28.707342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:28.707342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:28.707342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:28.707342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:28.707342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:28.707342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:28.707342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:28.707342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:28.707342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:28.707342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:28.708120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.708120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:28.708120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.708120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:28.708120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:28.708120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:28.708120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:28.708120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:28.708120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:28.708120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:28.708120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:28.708120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:28.708120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:28.708120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:28.708120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:28.709012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:28.709012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:28.709012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:28.709012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:11:28.709012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:28.709012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:28.709012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:28.709012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:28.709012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:28.709012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:28.709012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:28.709012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:28.709012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:28.709012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:28.709012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:28.709012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:28.709012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:28.709012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:28.709012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:28.709935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:28.709935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:28.709935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:28.709935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:28.709935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.709935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:28.721775 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:28.722188 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:28.722328 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:28.722463 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:11:28.723034 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:28.723210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.723815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:28.723815 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:28.723815 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:28.723815 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:28.723815 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:28.723815 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:28.723815 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.723815 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:28.724980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.724980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:11:28.724980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:28.724980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:28.724980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:28.724980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:28.724980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:28.724980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:28.724980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:28.724980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:28.724980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:28.724980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:11:28.724980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:28.724980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:11:28.724980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:11:28.725571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:28.725571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:11:28.725571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:28.725571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:11:28.725571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:11:28.725571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:28.725571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:11:28.725571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.725571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:11:28.726121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.726121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:28.726121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:28.726121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:28.726121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:28.726121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:11:28.726121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:11:28.726121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:11:28.726121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:28.726121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:28.726121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:28.726121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:28.726556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:28.726556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:28.726556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:28.726556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:28.726556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:28.726556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:28.726556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:28.726556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:28.726556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:28.726556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:28.726556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:28.726556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:28.726556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:28.726556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:28.727032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:28.727032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:28.727032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.727032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:28.727521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.727521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:28.727521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:28.727521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:28.727521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:28.727521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:28.727521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:28.727521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:28.727521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:28.727521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:28.727521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:28.727521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:28.727521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:28.727521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:28.727521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:28.727521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:28.728074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:11:28.728074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:11:28.728074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:11:28.728074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:28.728074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:28.728074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:28.728074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:28.728074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:28.728074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:28.728074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:28.728074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:28.728074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:28.728074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:28.728074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:28.728074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:28.728074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:28.728074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:28.728074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:28.728715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:28.728715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:28.728715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:28.728715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:28.728715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:28.728715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.728715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:28.736342 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:28.736654 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:28.736813 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:28.736966 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:28.737592 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:28.737929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.738581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:28.738581 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:28.738581 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:28.738581 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:28.738581 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:28.738581 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:28.738581 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.738581 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:28.739736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.739736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:11:28.739736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:28.739736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:28.739736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:28.739736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:28.739736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:28.739736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:28.739736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:28.739736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:28.739736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:28.739736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:11:28.739736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:28.739736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:11:28.739736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:11:28.740294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:28.740294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:11:28.740294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:28.740294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:11:28.740294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:11:28.740294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:28.740294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:11:28.740294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.740294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:11:28.741016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.741016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:28.741016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:28.741016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:28.741016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:28.741016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:11:28.741016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:11:28.741016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:28.741016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:28.741016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:28.741016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:28.741423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:28.741423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:28.741423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:28.741423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:28.741423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:28.741423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:28.741423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:28.741423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:28.741423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:28.741423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:28.741423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:28.741423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:28.741423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:28.741423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:28.741858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:28.741858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:28.741858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.741858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:28.742250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.742250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:28.742250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:28.742250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:28.742250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:28.742250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:28.742250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:28.742250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:28.742250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:28.742250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:28.742250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:28.742250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:28.742250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:28.742250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:28.742250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:28.742250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:28.742250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:11:28.742793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:11:28.742793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:28.742793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:28.742793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:28.742793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:28.742793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:28.742793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:28.742793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:28.742793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:28.742793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:28.742793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:28.742793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:28.742793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:28.742793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:28.742793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:28.742793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:28.742793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:28.742793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:28.742793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:28.743371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:28.743371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:28.743371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.743371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:28.743967 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:28.744323 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:28.744441 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:28.744639 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:28.745894 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:28.746147 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:28.746290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.747143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:28.747143 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:28.747143 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:28.747143 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:28.747143 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:28.747143 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:28.747143 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.747143 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:28.748428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.748428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:28.748428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:28.748428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:28.748428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:28.748428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:11:28.748428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:28.748428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:28.748428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:28.748428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:28.748428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:28.748428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:28.748876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:28.748876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:28.748876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:28.748876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:28.748876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:28.748876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:28.748876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:28.748876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:28.748876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:28.748876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:28.748876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:28.748876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:28.748876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:28.749273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.749273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:28.749704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.749704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:28.749704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:28.749704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:28.749704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:28.749704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:28.749704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:28.749704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:28.749704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:28.749704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:28.749704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:28.749704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:28.749704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:28.749704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:28.749704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:28.749704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:28.750227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:11:28.750227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:28.750227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:28.750227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:28.750227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:28.750227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:28.750227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:28.750227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:28.750227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:28.750227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:28.750227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:28.750227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:28.750227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:28.750227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:28.750227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:28.750227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:28.750227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:28.750227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:28.750227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:28.750822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:28.750822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:28.750822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:32.923430 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:32.925639 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:32.925922 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:32.927571 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:32.962191 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:32.962610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:32.963420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:32.963420 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:32.963420 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:32.963420 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:32.963420 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:32.963420 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:32.963420 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:32.963420 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:32.964224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:32.964224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:32.964224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:32.964224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:32.964224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:32.964224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:11:32.964224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:32.964224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:32.964224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:32.964224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:32.964224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:32.964224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:32.964880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:32.964880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:32.964880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:32.964880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:32.964880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:32.964880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:32.964880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:32.964880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:32.964880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:32.964880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:32.964880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:32.964880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:32.964880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:32.965457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:32.965457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:32.965457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:32.965457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:32.965457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:32.965457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:32.965457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:32.965457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:32.965457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:32.965457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:32.965457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:32.965457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:32.965457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:32.965457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:32.965457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:32.966196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:32.966196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:32.966196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:32.966196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:11:32.966196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:32.966196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:32.966196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:32.966196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:32.966196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:32.966196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:32.966196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:32.966196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:32.966196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:32.966196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:32.966196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:32.966196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:32.966196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:32.966196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:32.966196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:32.967142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:32.967142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:32.967142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:32.967142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:32.967142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:32.967142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:33.702319 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:33.702744 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:33.702974 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:33.703961 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:33.704297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.705174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:33.705174 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:33.705174 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:33.705174 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:33.705174 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:33.705174 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:33.705174 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.705174 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:33.706829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.706829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:33.706829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:33.706829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:33.706829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:33.706829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:11:33.706829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:33.706829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:33.706829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:33.706829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:33.706829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:33.706829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:33.707433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:33.707433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:33.707433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:33.707433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:33.707433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:33.707433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:33.707433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:33.707433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:33.707433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:33.707433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:33.707433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:33.707433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:33.707433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:33.708170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.708170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:33.708170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.708170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:33.708170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:33.708170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:33.708170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:33.708170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:33.708170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:33.708170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:33.708170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:33.708170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:33.708170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:33.708170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:33.708170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:33.709112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:33.709112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:33.709112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:33.709112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:11:33.709112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:33.709112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:33.709112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:33.709112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:33.709112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:33.709112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:33.709112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:33.709112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:33.709112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:33.709112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:33.709112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:33.709112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:33.709112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:33.709112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:33.709112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:33.710164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:33.710164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:33.710164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:33.710164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:33.710164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.710164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:33.722204 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:33.722543 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:33.722818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:33.723009 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:11:33.723401 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:33.723609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.724199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:33.724199 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:33.724199 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:33.724199 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:33.724199 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:33.724199 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:33.724199 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.724199 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:33.725331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.725331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:11:33.725331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:33.725331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:33.725331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:33.725331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:33.725331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:33.725331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:33.725331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:33.725331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:33.725331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:33.725331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:11:33.725331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:33.725331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:11:33.725331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:11:33.726119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:33.726119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:11:33.726119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:33.726119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:11:33.726119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:11:33.726119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:33.726119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:11:33.726119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.726119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:11:33.726662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.726662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:33.726662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:33.726662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:33.726662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:33.726662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:11:33.726662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:11:33.726662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:11:33.726662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:33.726662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:33.726662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:33.726662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:33.727421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:33.727421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:33.727421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:33.727421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:33.727421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:33.727421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:33.727421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:33.727421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:33.727421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:33.727421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:33.727421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:33.727421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:33.727421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:33.727421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:33.727879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:33.727879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:33.727879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.727879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:33.728151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.728151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:33.728151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:33.728151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:33.728151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:33.728151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:33.728151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:33.728151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:33.728151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:33.728151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:33.728151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:33.728151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:33.728151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:33.728151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:33.728151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:33.728151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:33.728631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:11:33.728631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:11:33.728631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:11:33.728631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:33.728631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:33.728631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:33.728631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:33.728631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:33.728631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:33.728631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:33.728631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:33.728631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:33.728631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:33.728631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:33.728631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:33.728631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:33.728631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:33.728631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:33.729174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:33.729174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:33.729174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:33.729174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:33.729174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:33.729174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.729174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:33.736702 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:33.737010 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:33.737190 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:33.737480 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:33.737928 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:33.738164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.738884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:33.738884 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:33.738884 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:33.738884 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:33.738884 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:33.738884 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:33.738884 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.738884 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:33.740261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.740261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:11:33.740261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:33.740261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:33.740261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:33.740261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:33.740261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:33.740261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:33.740261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:33.740261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:33.740261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:33.740261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:11:33.740261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:33.740261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:11:33.740261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:11:33.741010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:33.741010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:11:33.741010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:33.741010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:11:33.741010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:11:33.741010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:33.741010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:11:33.741010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.741010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:11:33.741960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.741960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:33.741960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:33.741960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:33.741960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:33.741960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:11:33.741960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:11:33.741960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:33.741960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:33.741960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:33.741960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:33.742520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:33.742520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:33.742520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:33.742520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:33.742520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:33.742520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:33.742520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:33.742520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:33.742520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:33.742520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:33.742520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:33.742520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:33.742520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:33.742520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:33.743222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:33.743222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:33.743222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.743222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:33.743747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.743747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:33.743747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:33.743747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:33.743747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:33.743747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:33.743747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:33.743747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:33.743747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:33.743747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:33.743747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:33.743747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:33.743747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:33.743747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:33.743747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:33.743747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:33.743747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:11:33.744630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:11:33.744630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:33.744630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:33.744630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:33.744630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:33.744630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:33.744630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:33.744630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:33.744630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:33.744630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:33.744630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:33.744630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:33.744630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:33.744630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:33.744630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:33.744630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:33.744630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:33.744630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:33.744630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:33.745608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:33.745608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:33.745608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.745608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:33.745608 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:33.745608 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:33.745608 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:33.745608 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:33.746039 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:33.746039 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:33.746039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.746039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:33.746039 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:33.746039 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:33.746039 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:33.746039 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:33.746039 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:33.746039 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.746039 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:33.747034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.747034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:33.747034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:33.747034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:33.747034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:33.747034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:11:33.747034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:33.747034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:33.747034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:33.747034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:33.747034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:33.747034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:33.747569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:33.747569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:33.747569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:33.747569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:33.747569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:33.747569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:33.747569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:33.747569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:33.747569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:33.747569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:33.747569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:33.747569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:33.747569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:33.747974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.747974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:33.748057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.748057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:33.748057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:33.748057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:33.748057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:33.748057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:33.748057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:33.748057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:33.748057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:33.748057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:33.748057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:33.748057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:33.748057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:33.748057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:33.748057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:33.748057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:33.748527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:11:33.748527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:33.748527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:33.748527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:33.748527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:33.748527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:33.748527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:33.748527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:33.748527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:33.748527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:33.748527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:33.748527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:33.748527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:33.748527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:33.748527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:33.748527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:33.748527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:33.748527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:33.748527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:33.749105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:33.749105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:33.749105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:37.924225 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:37.925388 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:37.925388 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:37.926340 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:37.959165 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:37.959470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:37.960647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:37.960647 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:37.960647 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:37.960647 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:37.960647 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:37.960647 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:37.960647 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:37.960647 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:37.961865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:37.961865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:37.961865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:37.961865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:37.961865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:37.961865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:11:37.961865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:37.961865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:37.961865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:37.961865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:37.961865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:37.961865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:37.963357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:37.963357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:37.963357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:37.963357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:37.963357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:37.963357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:37.963357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:37.963357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:37.963357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:37.963357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:37.963357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:37.963357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:37.963357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:37.965358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:37.965358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:37.965358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:37.965358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:37.965358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:37.965358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:37.965358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:37.965358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:37.965358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:37.965358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:37.965358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:37.965358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:37.965358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:37.965358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:37.965358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:37.966571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:37.966571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:37.966571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:37.966571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:11:37.966571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:37.966571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:37.966571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:37.966571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:37.966571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:37.966571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:37.966571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:37.966571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:37.966571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:37.966571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:37.966571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:37.966571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:37.966571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:37.966571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:37.966571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:37.967996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:37.967996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:37.967996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:37.967996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:37.967996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:37.967996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:38.702981 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:38.703370 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:38.703554 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:38.705947 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:38.705947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.706211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:38.706211 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:38.706211 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:38.706211 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:38.706211 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:38.706211 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:38.706211 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.706211 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:38.707700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.707700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:38.707700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:38.707700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:38.707700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:38.707700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:11:38.707700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:38.707700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:38.707700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:38.707700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:38.707700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:38.707700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:38.708074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:38.708074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:38.708074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:38.708074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:38.708074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:38.708074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:38.708074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:38.708074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:38.708074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:38.708074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:38.708074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:38.708074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:38.708074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:38.708408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.708408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:38.709457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.709457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:38.709457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:38.709457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:38.709457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:38.709457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:38.709457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:38.709457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:38.709457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:38.709457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:38.709457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:38.709457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:38.709457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:38.709457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:38.709457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:38.709457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:38.710508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:11:38.710508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:38.710508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:38.710508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:38.710508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:38.710508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:38.710508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:38.710508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:38.710508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:38.710508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:38.710508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:38.710508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:38.710508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:38.710508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:38.710508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:38.710508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:38.710508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:38.710508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:38.710508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:38.711275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:38.711275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.711275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:38.722600 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:38.723041 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:38.723180 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:38.723284 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:11:38.723892 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:38.724482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.725423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:38.725423 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:38.725423 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:38.725423 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:38.725423 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:38.725423 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:38.725423 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.725423 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:38.726188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.726188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:11:38.726188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:38.726188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:38.726188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:38.726188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:38.726188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:38.726188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:38.726188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:38.726188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:38.726188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:38.726188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:11:38.726188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:38.726188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:11:38.726188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:11:38.727427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:38.727427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:11:38.727427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:38.727427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:11:38.727427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:11:38.727427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:38.727427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:11:38.727427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.727427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:11:38.727427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.727427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:38.727427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:38.727427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:38.727427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:38.728718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:11:38.728718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:11:38.728718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:11:38.728718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:38.728718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:38.728718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:38.728718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:38.728718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:38.728718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:38.728718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:38.728718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:38.728718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:38.728718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:38.729392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:38.729392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:38.729392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:38.729392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:38.729392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:38.729392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:38.729392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:38.729392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:38.729392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:38.729392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:38.729392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.729392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:38.729986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.729986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:38.729986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:38.729986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:38.729986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:38.729986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:38.729986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:38.729986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:38.729986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:38.729986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:38.729986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:38.729986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:38.729986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:38.729986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:38.729986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:38.729986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:38.730429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:11:38.730429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:11:38.730429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:11:38.730429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:38.730429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:38.730429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:38.730429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:38.730429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:38.730429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:38.730429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:38.730429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:38.730429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:38.730429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:38.730429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:38.730429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:38.730429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:38.730429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:38.730429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:38.731612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:38.731612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:38.731612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:38.731612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:38.731612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:38.731612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.731612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:38.736964 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:38.737159 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:38.737271 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:38.737582 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:38.738592 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:38.738592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.738811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:38.738811 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:38.738811 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:38.738811 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:38.738811 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:38.738811 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:38.738811 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.738811 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:38.739462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.739462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:11:38.739462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:38.739462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:38.739462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:38.739462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:38.739462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:38.739462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:38.739462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:38.739462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:38.739462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:38.739462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:11:38.739462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:38.739462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:11:38.739462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:11:38.740320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:38.740320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:11:38.740320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:38.740320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:11:38.740320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:11:38.740320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:38.740320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:11:38.740320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.740320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:11:38.740320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.740320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:38.740320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:38.740320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:38.740320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:38.741162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:11:38.741162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:11:38.741162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:38.741162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:38.741162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:38.741162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:38.741162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:38.741162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:38.741162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:38.741162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:38.741162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:38.741162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:38.741162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:38.741696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:38.741696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:38.741696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:38.741696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:38.741696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:38.741696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:38.741696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:38.741696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:38.741696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:38.741696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.741696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:38.741696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.742016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:38.742016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:38.742016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:38.742016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:38.742016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:38.742016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:38.742016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:38.742016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:38.742016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:38.742016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:38.742016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:38.742016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:38.742016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:38.742016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:38.742016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:38.742016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:11:38.742016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:11:38.742016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:38.742016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:38.742509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:38.742509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:38.742509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:38.742509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:38.742509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:38.742509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:38.742509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:38.742509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:38.742509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:38.742509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:38.742509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:38.742509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:38.742509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:38.742509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:38.742509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:38.742509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:38.742509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:38.742509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:38.742509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.742509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:38.744561 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:38.744697 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:38.744820 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:38.744963 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:38.745277 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:38.745634 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:38.745820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.746282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:38.746282 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:38.746282 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:38.746282 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:38.746282 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:38.746282 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:38.746282 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.746282 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:38.747079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.747079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:38.747079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:38.747079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:38.747079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:38.747079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:11:38.747079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:38.747079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:38.747079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:38.747079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:38.747079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:38.747079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:38.747532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:38.747532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:38.747532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:38.747532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:38.747532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:38.747532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:38.747532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:38.747532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:38.747532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:38.747532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:38.747532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:38.747532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:38.747532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:38.748007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.748007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:38.748088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.748088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:38.748088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:38.748088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:38.748088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:38.748088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:38.748088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:38.748088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:38.748088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:38.748088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:38.748088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:38.748088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:38.748088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:38.748088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:38.748088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:38.748088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:38.748666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:11:38.748666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:38.748666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:38.748666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:38.748666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:38.748666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:38.748666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:38.748666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:38.748666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:38.748666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:38.748666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:38.748666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:38.748666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:38.748666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:38.748666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:38.748666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:38.748666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:38.748666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:38.748666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:38.749363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:38.749363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:38.749363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:42.925061 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:42.925640 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:42.925640 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:42.926903 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:42.968576 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:42.968875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:42.969722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:42.969722 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:42.969722 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:42.969722 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:42.969722 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:42.969722 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:42.969722 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:42.969722 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:42.971463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:42.971463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:42.971463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:42.971463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:42.971463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:42.971463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:11:42.971463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:42.971463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:42.971463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:42.971463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:42.971463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:42.971463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:42.972108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:42.972108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:42.972108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:42.972108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:42.972108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:42.972108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:42.972108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:42.972108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:42.972108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:42.972108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:42.972108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:42.972108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:42.972108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:42.972665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:42.972665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:42.974258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:42.974258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:42.974258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:42.974258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:42.974258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:42.974258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:42.974258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:42.974258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:42.974258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:42.974258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:42.974258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:42.974258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:42.974258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:42.974258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:42.974258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:42.974258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:42.975125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:11:42.975125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:42.975125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:42.975125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:42.975125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:42.975125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:42.975125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:42.975125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:42.975125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:42.975125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:42.975125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:42.975125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:42.975125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:42.975125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:42.975125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:42.975125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:42.975125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:42.975125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:42.975125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:42.975981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:42.975981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:42.975981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:43.703837 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:43.704690 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:43.704954 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:43.706138 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:43.706486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.707454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:43.707454 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:43.707454 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:43.707454 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:43.707454 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:43.707454 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:43.707454 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.707454 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:43.709301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.709301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:43.709301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:43.709301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:43.709301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:43.709301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:11:43.709301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:43.709301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:43.709301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:43.709301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:43.709301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:43.709301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:43.709592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:43.709592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:43.709592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:43.709592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:43.709592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:43.709592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:43.709592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:43.709592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:43.709592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:43.709592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:43.709592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:43.709592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:43.709592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:43.709908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.709908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:43.711530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.711530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:43.711530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:43.711530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:43.711530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:43.711530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:43.711530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:43.711530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:43.711530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:43.711530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:43.711530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:43.711530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:43.711530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:43.711530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:43.711530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:43.711530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:43.711912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:11:43.711912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:43.711912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:43.711912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:43.711912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:43.711912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:43.711912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:43.711912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:43.711912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:43.711912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:43.711912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:43.711912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:43.711912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:43.711912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:43.711912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:43.711912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:43.711912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:43.711912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:43.711912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:43.712354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:43.712354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.712354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:43.725335 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:43.725768 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:43.725978 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:43.726181 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:11:43.726784 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:43.727114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.727965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:43.727965 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:43.727965 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:43.727965 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:43.727965 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:43.727965 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:43.727965 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.727965 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:43.729478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.729478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:11:43.729478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:43.729478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:43.729478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:43.729478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:43.729478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:43.729478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:43.729478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:43.729478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:43.729478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:43.729478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:11:43.729478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:43.729478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:11:43.729478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:11:43.729863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:43.729863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:11:43.729863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:43.729863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:11:43.729863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:11:43.729863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:43.729863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:11:43.729863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.729863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:11:43.731018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.731018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:43.731018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:43.731018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:43.731018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:43.731018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:11:43.731018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:11:43.731018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:11:43.731018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:43.731018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:43.731018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:43.731018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:43.731311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:43.731311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:43.731311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:43.731311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:43.731311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:43.731311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:43.731311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:43.731311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:43.731311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:43.731311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:43.731311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:43.731311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:43.731311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:43.731311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:43.731617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:43.731617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:43.731617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.731617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:43.732825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.732825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:43.732825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:43.732825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:43.732825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:43.732825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:43.732825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:43.732825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:43.732825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:43.732825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:43.732825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:43.732825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:43.732825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:43.732825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:43.732825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:43.732825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:43.733308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:11:43.733308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:11:43.733308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:11:43.733308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:43.733308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:43.733308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:43.733308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:43.733308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:43.733308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:43.733308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:43.733308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:43.733308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:43.733308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:43.733308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:43.733308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:43.733308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:43.733308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:43.733308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:43.733716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:43.733716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:43.733716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:43.733716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:43.733716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:43.733716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.733716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:43.737325 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:43.737525 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:43.737641 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:43.737782 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:43.738256 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:43.738656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.739383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:43.739383 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:43.739383 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:43.739383 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:43.739383 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:43.739383 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:43.739383 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.739383 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:43.740316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.740316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:11:43.740316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:43.740316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:43.740316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:43.740316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:43.740316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:43.740316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:43.740316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:43.740316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:43.740316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:43.740316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:11:43.740316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:43.740316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:11:43.740316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:11:43.740651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:43.740651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:11:43.740651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:43.740651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:11:43.740651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:11:43.740651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:43.740651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:11:43.740651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.740651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:11:43.741361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.741361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:43.741361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:43.741361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:43.741361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:43.741361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:11:43.741361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:11:43.741361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:43.741361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:43.741361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:43.741361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:43.741892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:43.741892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:43.741892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:43.741892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:43.741892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:43.741892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:43.741892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:43.741892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:43.741892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:43.741892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:43.741892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:43.741892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:43.741892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:43.741892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:43.742471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:43.742471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:43.742471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.742471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:43.742835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.742835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:43.742835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:43.742835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:43.742835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:43.742835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:43.742835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:43.742835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:43.742835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:43.742835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:43.742835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:43.742835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:43.742835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:43.742835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:43.742835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:43.742835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:43.742835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:11:43.743453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:11:43.743453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:43.743453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:43.743453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:43.743453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:43.743453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:43.743453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:43.743453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:43.743453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:43.743453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:43.743453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:43.743453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:43.743453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:43.743453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:43.743453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:43.743453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:43.743453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:43.743453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:43.743453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:43.743896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:43.743896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:43.743896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.743896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:43.744863 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:43.745243 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:43.745423 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:43.745654 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:43.746162 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:43.746511 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:43.746696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.747528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:43.747528 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:43.747528 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:43.747528 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:43.747528 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:43.747528 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:43.747528 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.747528 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:43.749110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.749110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:43.749110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:43.749110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:43.749110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:43.749110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:11:43.749110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:43.749110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:43.749110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:43.749110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:43.749110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:43.749110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:43.749636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:43.749636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:43.749636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:43.749636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:43.749636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:43.749636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:43.749636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:43.749636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:43.749636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:43.749636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:43.749636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:43.749636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:43.749636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:43.750217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.750217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:43.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:43.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:43.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:43.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:43.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:43.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:43.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:43.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:43.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:43.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:43.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:43.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:43.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:43.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:43.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:43.750757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:11:43.750757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:43.750757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:43.750757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:43.750757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:43.750757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:43.750757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:43.750757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:43.750757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:43.750757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:43.750757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:43.750757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:43.750757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:43.750757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:43.750757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:43.750757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:43.750757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:43.750757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:43.750757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:43.751171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:43.751171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:43.751171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:47.925659 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:47.926212 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:47.926212 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:47.926826 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:47.956581 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:47.956909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:47.957564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:47.957564 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:47.957564 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:47.957564 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:47.957564 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:47.957564 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:47.957564 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:47.957564 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:47.959113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:47.959113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:47.959113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:47.959113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:47.959113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:47.959113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:11:47.959113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:47.959113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:47.959113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:47.959113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:47.959113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:47.959113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:47.959638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:47.959638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:47.959638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:47.959638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:47.959638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:47.959638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:47.959638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:47.959638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:47.959638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:47.959638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:47.959638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:47.959638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:47.959638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:47.960139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:47.960139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:47.960139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:47.960139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:47.960139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:47.960139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:47.960139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:47.960139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:47.960139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:47.960139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:47.960139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:47.960139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:47.960139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:47.960139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:47.960139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:47.960753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:47.960753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:47.960753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:47.960753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:11:47.960753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:47.960753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:47.960753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:47.960753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:47.960753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:47.960753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:47.960753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:47.960753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:47.960753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:47.960753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:47.960753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:47.960753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:47.960753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:47.960753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:47.960753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:47.961484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:47.961484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:47.961484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:47.961484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:47.961484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:47.961484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:48.704567 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:48.705269 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:48.705537 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:48.706410 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:48.706780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.707734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:48.707734 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:48.707734 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:48.707734 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:48.707734 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:48.707734 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:48.707734 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.707734 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:48.709274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.709274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:48.709274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:48.709274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:48.709274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:48.709274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:11:48.709274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:48.709274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:48.709274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:48.709274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:48.709274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:48.709274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:48.709873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:48.709873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:48.709873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:48.709873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:48.709873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:48.709873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:48.709873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:48.709873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:48.709873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:48.709873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:48.709873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:48.709873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:48.709873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:48.710391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.710391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:48.710769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.710769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:48.710769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:48.710769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:48.710769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:48.710769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:48.710769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:48.710769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:48.710769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:48.710769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:48.710769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:48.710769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:48.710769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:48.710769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:48.710769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:48.710769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:48.711270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:11:48.711270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:48.711270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:48.711270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:48.711270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:48.711270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:48.711270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:48.711270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:48.711270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:48.711270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:48.711270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:48.711270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:48.711270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:48.711270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:48.711270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:48.711270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:48.711270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:48.711270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:48.711270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:48.711846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:48.711846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.711846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:48.725770 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:48.726092 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:48.726277 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:48.726457 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:11:48.726795 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:48.727081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.727923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:48.727923 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:48.727923 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:48.727923 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:48.727923 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:48.727923 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:48.727923 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.727923 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:48.729330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.729330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:11:48.729330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:48.729330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:48.729330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:48.729330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:48.729330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:48.729330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:48.729330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:48.729330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:48.729330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:48.729330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:11:48.729330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:48.729330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:11:48.729330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:11:48.730408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:48.730408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:11:48.730408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:48.730408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:11:48.730408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:11:48.730408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:48.730408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:11:48.730408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.730408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:11:48.732274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.732274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:48.732274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:48.732274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:48.732274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:48.732274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:11:48.732274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:11:48.732274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:11:48.732274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:48.732274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:48.732274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:48.732274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:48.733159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:48.733159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:48.733159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:48.733159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:48.733159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:48.733159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:48.733159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:48.733159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:48.733159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:48.733159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:48.733159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:48.733159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:48.733159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:48.733159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:48.733621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:48.733621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:48.733621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.733621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:48.733621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.733621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:48.733621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:48.733621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:48.733621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:48.733621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:48.733621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:48.733621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:48.733621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:48.733621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:48.734058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:48.734058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:48.734058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:48.734058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:48.734058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:48.734058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:48.734058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:11:48.734058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:11:48.734058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:11:48.734058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:48.734058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:48.734058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:48.734058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:48.734058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:48.734058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:48.734058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:48.734058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:48.734058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:48.734058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:48.734531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:48.734531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:48.734531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:48.734531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:48.734531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:48.734531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:48.734531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:48.734531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:48.734531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:48.734531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:48.734531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.734531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:48.737652 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:48.737923 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:48.738028 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:48.738160 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:48.738357 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:48.738557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.738957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:48.738957 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:48.738957 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:48.738957 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:48.738957 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:48.738957 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:48.738957 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.738957 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:48.739550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.739550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:11:48.739550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:48.739550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:48.739550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:48.739550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:48.739550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:48.739550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:48.739550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:48.739550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:48.739550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:48.739550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:11:48.739550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:48.739550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:11:48.739550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:11:48.739971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:48.739971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:11:48.739971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:48.739971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:11:48.739971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:11:48.739971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:48.739971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:11:48.739971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.739971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:11:48.740444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.740444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:48.740444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:48.740444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:48.740444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:48.740444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:11:48.740444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:11:48.740444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:48.740444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:48.740444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:48.740444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:48.740831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:48.740831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:48.740831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:48.740831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:48.740831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:48.740831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:48.740831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:48.740831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:48.740831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:48.740831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:48.740831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:48.740831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:48.740831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:48.740831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:48.741230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:48.741230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:48.741230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.741230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:48.741512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.741512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:48.741512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:48.741512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:48.741512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:48.741512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:48.741512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:48.741512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:48.741512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:48.741512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:48.741512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:48.741512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:48.741512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:48.741512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:48.741512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:48.741512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:48.741512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:11:48.741975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:11:48.741975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:48.741975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:48.741975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:48.741975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:48.741975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:48.741975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:48.741975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:48.741975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:48.741975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:48.741975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:48.741975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:48.741975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:48.741975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:48.741975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:48.741975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:48.741975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:48.741975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:48.741975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:48.742332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:48.742332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:48.742332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.742332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:48.745181 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:48.745475 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:48.745615 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:48.745865 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:48.746133 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:48.747431 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:48.747431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.747431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:48.747431 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:48.747431 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:48.747431 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:48.747431 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:48.747431 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:48.747431 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.747431 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:48.747999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.747999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:48.747999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:48.747999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:48.747999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:48.747999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:11:48.747999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:48.747999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:48.747999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:48.747999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:48.747999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:48.747999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:48.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:48.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:48.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:48.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:48.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:48.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:48.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:48.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:48.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:48.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:48.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:48.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:48.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:48.748747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.748747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:48.749042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.749042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:48.749042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:48.749042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:48.749042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:48.749042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:48.749042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:48.749042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:48.749042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:48.749042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:48.749042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:48.749042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:48.749042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:48.749042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:48.749042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:48.749042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:48.749496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:11:48.749496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:48.749496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:48.749496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:48.749496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:48.749496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:48.749496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:48.749496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:48.749496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:48.749496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:48.749496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:48.749496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:48.749496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:48.749496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:48.749496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:48.749496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:48.749496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:48.749496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:48.749496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:48.750040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:48.750040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:48.750040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:52.926871 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:52.928047 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:52.928314 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:52.929640 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:52.964993 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:52.965204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:52.965832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:52.965832 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:52.965832 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:52.965832 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:52.965832 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:52.965832 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:52.965832 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:52.965832 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:52.966825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:52.966825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:52.966825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:52.966825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:52.966825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:52.966825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:11:52.966825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:52.966825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:52.966825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:52.966825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:52.966825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:52.966825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:52.967512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:52.967512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:52.967512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:52.967512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:52.967512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:52.967512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:52.967512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:52.967512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:52.967512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:52.967512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:52.967512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:52.967512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:52.967512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:52.968291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:52.968291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:52.968291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:52.968291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:52.968291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:52.968291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:52.968291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:52.968291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:52.968291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:52.968291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:52.968291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:52.968291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:52.968291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:52.968291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:52.968291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:52.969314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:52.969314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:52.969314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:52.969314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:11:52.969314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:52.969314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:52.969314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:52.969314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:52.969314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:52.969314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:52.969314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:52.969314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:52.969314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:52.969314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:52.969314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:52.969314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:52.969314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:52.969314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:52.969314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:52.970421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:52.970421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:52.970421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:52.970421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:52.970421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:52.970421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:53.705351 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:53.706023 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:53.706200 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:53.707107 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:53.707432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.708324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:53.708324 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:53.708324 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:53.708324 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:53.708324 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:53.708324 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:53.708324 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.708324 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:53.710204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.710204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:53.710204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:53.710204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:53.710204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:53.710204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:11:53.710204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:53.710204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:53.710204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:53.710204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:53.710204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:53.710204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:53.711097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:53.711097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:53.711097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:53.711097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:53.711097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:53.711097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:53.711097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:53.711097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:53.711097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:53.711097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:53.711097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:53.711097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:53.711097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:53.711897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.711897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:53.711897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.711897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:53.711897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:53.711897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:53.711897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:53.711897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:53.711897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:53.711897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:53.711897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:53.711897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:53.711897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:53.711897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:53.711897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:53.712746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:53.712746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:53.712746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:53.712746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:11:53.712746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:53.712746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:53.712746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:53.712746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:53.712746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:53.712746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:53.712746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:53.712746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:53.712746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:53.712746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:53.712746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:53.712746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:53.712746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:53.712746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:53.712746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:53.713978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:53.713978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:53.713978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:53.713978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:53.713978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.713978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:53.726267 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:53.726638 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:53.726809 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:53.726954 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:11:53.727408 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:53.727783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.728755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:53.728755 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:53.728755 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:53.728755 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:53.728755 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:53.728755 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:53.728755 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.728755 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:53.731114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.731114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:11:53.731114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:53.731114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:53.731114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:53.731114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:53.731114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:53.731114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:53.731114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:53.731114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:53.731114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:53.731114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:11:53.731114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:53.731114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:11:53.731114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:11:53.732041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:53.732041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:11:53.732041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:53.732041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:11:53.732041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:11:53.732041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:53.732041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:11:53.732041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.732041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:11:53.732041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.732041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:53.732041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:53.732041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:53.732041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:53.732701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:11:53.732701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:11:53.732701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:11:53.732701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:53.732701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:53.732701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:53.732701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:53.732701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:53.732701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:53.732701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:53.732701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:53.732701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:53.732701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:53.733334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:53.733334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:53.733334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:53.733334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:53.733334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:53.733334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:53.733334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:53.733334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:53.733334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:53.733334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:53.733334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.733334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:53.733932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.733932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:53.733932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:53.733932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:53.733932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:53.733932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:53.733932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:53.733932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:53.733932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:53.733932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:53.733932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:53.733932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:53.733932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:53.733932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:53.733932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:53.733932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:53.734589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:11:53.734589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:11:53.734589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:11:53.734589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:53.734589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:53.734589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:53.734589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:53.734589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:53.734589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:53.734589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:53.734589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:53.734589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:53.734589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:53.734589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:53.734589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:53.734589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:53.734589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:53.734589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:53.735656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:53.735656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:53.735656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:53.735656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:53.735656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:53.735656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.735656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:53.738067 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:53.738348 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:53.738465 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:53.738600 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:53.739115 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:53.739508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.740262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:53.740262 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:53.740262 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:53.740262 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:53.740262 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:53.740262 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:53.740262 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.740262 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:53.740999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.740999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:11:53.740999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:53.740999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:53.740999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:53.740999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:53.740999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:53.740999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:53.740999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:53.740999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:53.740999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:53.740999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:11:53.740999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:53.740999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:11:53.740999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:11:53.741294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:53.741294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:11:53.741294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:53.741294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:11:53.741294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:11:53.741294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:53.741294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:11:53.741294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.741294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:11:53.742065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.742065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:53.742065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:53.742065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:53.742065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:53.742065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:11:53.742065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:11:53.742065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:53.742065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:53.742065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:53.742065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:53.742299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:53.742299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:53.742299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:53.742299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:53.742299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:53.742299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:53.742299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:53.742299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:53.742299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:53.742299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:53.742299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:53.742299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:53.742299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:53.742299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:53.742549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:53.742549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:53.742549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.742549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:53.743126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.743126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:53.743126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:53.743126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:53.743126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:53.743126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:53.743126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:53.743126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:53.743126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:53.743126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:53.743126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:53.743126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:53.743126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:53.743126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:53.743126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:53.743126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:53.743126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:11:53.743454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:11:53.743454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:53.743454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:53.743454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:53.743454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:53.743454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:53.743454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:53.743454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:53.743454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:53.743454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:53.743454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:53.743454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:53.743454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:53.743454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:53.743454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:53.743454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:53.743454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:53.743454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:53.743454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:53.743843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:53.743843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:53.743843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.743843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:53.745651 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:53.746132 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:53.746281 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:53.746489 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:53.746948 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:53.747305 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:53.747495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.748197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:53.748197 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:53.748197 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:53.748197 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:53.748197 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:53.748197 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:53.748197 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.748197 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:53.749015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.749015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:53.749015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:53.749015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:53.749015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:53.749015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:11:53.749015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:53.749015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:53.749015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:53.749015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:53.749015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:53.749015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:53.749483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:53.749483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:53.749483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:53.749483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:53.749483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:53.749483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:53.749483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:53.749483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:53.749483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:53.749483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:53.749483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:53.749483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:53.749483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:53.749919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.749919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:53.750059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.750059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:53.750059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:53.750059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:53.750059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:53.750059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:53.750059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:53.750059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:53.750059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:53.750059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:53.750059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:53.750059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:53.750059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:53.750059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:53.750059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:53.750059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:53.750544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:11:53.750544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:53.750544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:53.750544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:53.750544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:53.750544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:53.750544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:53.750544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:53.750544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:53.750544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:53.750544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:53.750544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:53.750544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:53.750544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:53.750544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:53.750544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:53.750544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:53.750544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:53.750544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:53.751122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:53.751122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:53.751122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:57.928379 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:57.929474 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:57.929592 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:57.930825 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:57.962244 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:57.962627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:57.963581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:57.963581 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:57.963581 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:57.963581 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:57.963581 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:57.963581 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:57.963581 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:57.963581 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:57.964364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:57.964364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:57.964364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:57.964364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:57.964364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:57.964364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:11:57.964364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:57.964364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:57.964364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:57.964364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:57.964364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:57.964364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:57.964981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:57.964981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:57.964981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:57.964981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:57.964981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:57.964981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:57.964981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:57.964981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:57.964981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:57.964981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:57.964981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:57.964981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:57.964981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:57.965621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:57.965621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:57.965621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:57.965621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:57.965621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:57.965621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:57.965621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:57.965621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:57.965621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:57.965621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:57.965621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:57.965621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:57.965621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:57.965621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:57.965621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:57.966294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:57.966294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:57.966294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:57.966294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:11:57.966294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:57.966294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:57.966294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:57.966294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:57.966294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:57.966294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:57.966294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:57.966294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:57.966294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:57.966294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:57.966294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:57.966294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:57.966294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:57.966294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:57.966294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:57.967137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:57.967137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:57.967137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:57.967137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:57.967137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:57.967137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:58.706014 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:58.706633 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:58.706861 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:58.707879 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:58.708242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.709292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:58.709292 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:58.709292 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:58.709292 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:58.709292 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:58.709292 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:58.709292 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.709292 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:58.710869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.710869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:58.710869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:58.710869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:58.710869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:58.710869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:11:58.710869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:58.710869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:58.710869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:58.710869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:58.710869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:58.710869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:58.711479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:58.711479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:58.711479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:58.711479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:58.711479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:58.711479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:58.711479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:58.711479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:58.711479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:58.711479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:58.711479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:58.711479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:58.711479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:58.712119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.712119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:58.712784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.712784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:58.712784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:58.712784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:58.712784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:58.712784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:58.712784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:58.712784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:58.712784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:58.712784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:58.712784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:58.712784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:58.712784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:58.712784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:58.712784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:58.712784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:58.713517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:11:58.713517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:58.713517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:58.713517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:58.713517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:58.713517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:58.713517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:58.713517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:58.713517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:58.713517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:58.713517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:58.713517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:58.713517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:58.713517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:58.713517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:58.713517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:58.713517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:58.713517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:58.713517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:58.714436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:58.714436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.714436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:58.726855 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:58.727357 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:58.727618 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:58.727911 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:11:58.728562 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:58.728978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.729833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:58.729833 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:58.729833 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:58.729833 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:58.729833 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:58.729833 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:58.729833 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.729833 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:58.731073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.731073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:11:58.731073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:58.731073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:58.731073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:58.731073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:58.731073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:58.731073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:58.731073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:58.731073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:58.731073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:58.731073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:11:58.731073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:58.731073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:11:58.731073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:11:58.732134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:58.732134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:11:58.732134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:58.732134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:11:58.732134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:11:58.732134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:58.732134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:11:58.732134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.732134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:11:58.732598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.732598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:58.732598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:58.732598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:58.732598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:58.732598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:11:58.732598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:11:58.732598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:11:58.732598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:58.732598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:58.732598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:58.732598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:58.733195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:58.733195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:58.733195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:58.733195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:58.733195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:58.733195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:58.733195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:58.733195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:58.733195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:58.733195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:58.733195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:58.733195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:58.733195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:58.733195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:58.733879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:58.733879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:58.733879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.733879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:58.734272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.734272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:58.734272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:58.734272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:58.734272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:58.734272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:58.734272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:58.734272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:58.734272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:58.734272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:58.734272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:58.734272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:58.734272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:58.734272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:58.734272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:58.734272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:58.735206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:11:58.735206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:11:58.735206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:11:58.735206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:58.735206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:58.735206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:58.735206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:58.735206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:58.735206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:58.735206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:58.735206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:58.735206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:58.735206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:58.735206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:58.735206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:58.735206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:58.735206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:58.735206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:58.736098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:58.736098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:58.736098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:58.736098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:58.736098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:58.736098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.736098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:58.738697 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:58.739124 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:58.739296 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:58.739461 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:58.739969 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:58.740338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.741199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:58.741199 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:58.741199 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:58.741199 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:58.741199 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:58.741199 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:58.741199 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.741199 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:58.741954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.741954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:11:58.741954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:58.741954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:58.741954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:58.741954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:58.741954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:58.741954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:58.741954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:58.741954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:11:58.741954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:58.741954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:11:58.741954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:58.741954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:11:58.741954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:11:58.742450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:58.742450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:11:58.742450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:58.742450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:11:58.742450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:11:58.742450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:58.742450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:11:58.742450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.742450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:11:58.742945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.742945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:58.742945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:58.742945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:58.742945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:58.742945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:11:58.742945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:11:58.742945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:58.742945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:58.742945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:58.742945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:58.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:58.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:58.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:58.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:58.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:58.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:58.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:58.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:58.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:58.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:58.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:58.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:58.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:58.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:58.743747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:58.743747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:58.743747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.743747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:58.744103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.744103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:58.744103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:58.744103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:58.744103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:58.744103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:58.744103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:58.744103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:58.744103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:58.744103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:58.744103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:58.744103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:58.744103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:58.744103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:58.744103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:58.744103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:58.744103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:11:58.744911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:11:58.744911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:11:58.744911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:11:58.744911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:58.744911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:58.744911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:58.744911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:58.744911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:58.744911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:58.744911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:58.744911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:58.744911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:58.744911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:58.744911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:58.744911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:58.744911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:58.744911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:58.744911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:58.744911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:58.745788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:58.745788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:58.745788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.745788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:11:58.746189 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:11:58.746428 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:11:58.746522 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:11:58.746698 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:11:58.747160 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:58.747564 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:11:58.747841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.748591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:11:58.748591 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:58.748591 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:11:58.748591 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:11:58.748591 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:58.748591 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:11:58.748591 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.748591 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:11:58.749422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.749422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:11:58.749422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:58.749422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:11:58.749422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:58.749422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:11:58.749422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:58.749422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:58.749422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:58.749422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:58.749422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:58.749422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:58.749822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:11:58.749822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:58.749822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:11:58.749822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:58.749822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:11:58.749822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:11:58.749822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:58.749822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:11:58.749822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:58.749822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:11:58.749822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:11:58.749822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:58.749822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:11:58.750277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.750277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:11:58.750438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.750438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:11:58.750438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:11:58.750438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:11:58.750438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:11:58.750438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:11:58.750438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:58.750438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:11:58.750438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:11:58.750438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:58.750438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:11:58.750438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:11:58.750438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:11:58.750438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:11:58.750438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:11:58.750438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:11:58.750933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:11:58.750933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:11:58.750933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:58.750933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:58.750933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:11:58.750933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:11:58.750933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:11:58.750933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:11:58.750933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:11:58.750933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:11:58.750933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:11:58.750933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:11:58.750933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:11:58.750933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:11:58.750933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:11:58.750933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:11:58.750933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:11:58.750933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:11:58.750933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:11:58.751476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:11:58.751476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:11:58.751476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:02.929868 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:02.930590 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:02.930838 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:02.931864 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:02.933612 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=580,neutron:INSERT=1,neutron:UPDATE=2,neutron:DELETE=1 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:12:02.972402 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:02.972809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:02.973699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:02.973699 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:02.973699 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:02.973699 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:02.973699 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:02.973699 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:02.973699 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:02.973699 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:02.974480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:02.974480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:02.974480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:02.974480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:02.974480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:02.974480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:12:02.974480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:02.974480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:02.974480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:02.974480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:02.974480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:02.974480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:02.975185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:02.975185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:02.975185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:02.975185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:02.975185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:02.975185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:02.975185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:02.975185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:02.975185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:02.975185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:02.975185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:02.975185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:02.975185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:02.976022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:02.976022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:02.976022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:02.976022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:02.976022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:02.976022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:02.976022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:02.976022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:02.976022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:02.976022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:02.976022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:02.976022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:02.976022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:02.976022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:02.976022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:02.977018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:02.977018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:02.977018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:02.977018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:12:02.977018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:02.977018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:02.977018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:02.977018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:02.977018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:02.977018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:02.977018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:02.977018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:02.977018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:02.977018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:02.977018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:02.977018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:02.977018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:02.977018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:02.977018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:02.978231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:02.978231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:02.978231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:02.978231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:02.978231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:02.978231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:03.706806 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:03.707807 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:03.707911 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:03.708640 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:03.708938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.709601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:03.709601 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:03.709601 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:03.709601 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:03.709601 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:03.709601 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:03.709601 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.709601 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:03.710286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.710286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:03.710286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:03.710286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:03.710286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:03.710286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:12:03.710286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:03.710286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:03.710286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:03.710286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:03.710286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:03.710286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:03.710885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:03.710885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:03.710885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:03.710885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:03.710885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:03.710885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:03.710885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:03.710885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:03.710885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:03.710885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:03.710885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:03.710885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:03.710885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:03.711431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.711431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:03.711431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.711431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:03.711431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:03.711431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:03.711431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:03.711431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:03.711431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:03.711431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:03.711431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:03.711431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:03.711431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:03.711431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:03.711431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:03.712186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:03.712186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:03.712186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:03.712186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:12:03.712186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:03.712186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:03.712186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:03.712186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:03.712186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:03.712186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:03.712186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:03.712186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:03.712186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:03.712186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:03.712186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:03.712186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:03.712186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:03.712186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:03.712186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:03.713116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:03.713116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:03.713116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:03.713116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:03.713116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.713116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:03.727401 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:03.728867 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:03.728867 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:03.728867 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:12:03.728867 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:03.728867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.729478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:03.729478 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:03.729478 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:03.729478 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:03.729478 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:03.729478 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:03.729478 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.729478 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:03.729930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.729930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:12:03.729930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:03.729930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:03.729930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:03.729930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:03.729930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:03.729930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:03.729930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:03.729930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:03.729930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:03.729930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:12:03.729930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:03.729930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:12:03.729930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:12:03.730936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:03.730936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:12:03.730936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:03.730936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:12:03.730936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:12:03.730936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:03.730936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:12:03.730936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.730936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:12:03.730936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.730936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:03.730936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:03.730936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:03.730936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:03.731592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:12:03.731592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:12:03.731592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:12:03.731592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:03.731592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:03.731592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:03.731592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:03.731592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:03.731592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:03.731592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:03.731592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:03.731592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:03.731592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:03.732322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:03.732322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:03.732322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:03.732322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:03.732322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:03.732322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:03.732322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:03.732322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:03.732322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:03.732322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:03.732322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.732322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:03.732924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.732924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:03.732924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:03.732924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:03.732924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:03.732924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:03.732924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:03.732924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:03.732924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:03.732924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:03.732924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:03.732924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:03.732924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:03.732924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:03.732924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:03.732924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:03.733663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:12:03.733663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:12:03.733663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:12:03.733663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:03.733663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:03.733663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:03.733663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:03.733663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:03.733663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:03.733663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:03.733663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:03.733663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:03.733663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:03.733663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:03.733663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:03.733663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:03.733663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:03.733663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:03.734634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:03.734634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:03.734634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:03.734634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:03.734634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:03.734634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.734634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:03.739309 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:03.739562 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:03.739646 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:03.739764 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:03.740256 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:03.740667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.741368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:03.741368 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:03.741368 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:03.741368 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:03.741368 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:03.741368 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:03.741368 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.741368 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:03.742082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.742082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:12:03.742082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:03.742082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:03.742082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:03.742082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:03.742082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:03.742082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:03.742082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:03.742082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:03.742082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:03.742082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:12:03.742082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:03.742082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:12:03.742082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:12:03.742559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:03.742559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:12:03.742559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:03.742559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:12:03.742559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:12:03.742559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:03.742559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:12:03.742559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.742559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:12:03.742941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.742941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:03.742941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:03.742941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:03.742941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:03.742941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:12:03.742941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:12:03.742941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:03.742941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:03.742941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:03.742941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:03.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:03.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:03.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:03.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:03.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:03.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:03.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:03.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:03.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:03.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:03.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:03.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:03.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:03.743292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:03.743717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:03.743717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:03.743717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.743717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:03.743982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.743982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:03.743982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:03.743982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:03.743982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:03.743982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:03.743982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:03.743982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:03.743982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:03.743982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:03.743982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:03.743982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:03.743982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:03.743982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:03.743982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:03.743982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:03.743982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:12:03.744494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:12:03.744494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:03.744494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:03.744494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:03.744494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:03.744494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:03.744494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:03.744494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:03.744494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:03.744494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:03.744494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:03.744494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:03.744494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:03.744494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:03.744494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:03.744494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:03.744494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:03.744494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:03.744494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:03.745076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:03.745076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:03.745076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.745076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:03.746516 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:03.746804 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:03.746881 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:03.747047 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:03.747464 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:03.747891 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:03.748081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.748801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:03.748801 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:03.748801 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:03.748801 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:03.748801 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:03.748801 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:03.748801 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.748801 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:03.749597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.749597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:03.749597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:03.749597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:03.749597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:03.749597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:12:03.749597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:03.749597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:03.749597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:03.749597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:03.749597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:03.749597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:03.749999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:03.749999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:03.749999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:03.749999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:03.749999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:03.749999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:03.749999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:03.749999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:03.749999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:03.749999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:03.749999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:03.749999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:03.749999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:03.750367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.750367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:03.750538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.750538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:03.750538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:03.750538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:03.750538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:03.750538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:03.750538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:03.750538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:03.750538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:03.750538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:03.750538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:03.750538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:03.750538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:03.750538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:03.750538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:03.750538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:03.751063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:12:03.751063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:03.751063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:03.751063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:03.751063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:03.751063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:03.751063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:03.751063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:03.751063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:03.751063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:03.751063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:03.751063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:03.751063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:03.751063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:03.751063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:03.751063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:03.751063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:03.751063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:03.751063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:03.751619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:03.751619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:03.751619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:07.930499 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:07.931564 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:07.931564 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:07.933251 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:07.968523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:07.969068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:07.970048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:07.970048 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:07.970048 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:07.970048 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:07.970048 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:07.970048 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:07.970048 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:07.970048 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:07.971124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:07.971124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:07.971124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:07.971124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:07.971124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:07.971124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:12:07.971124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:07.971124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:07.971124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:07.971124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:07.971124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:07.971124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:07.971673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:07.971673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:07.971673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:07.971673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:07.971673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:07.971673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:07.971673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:07.971673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:07.971673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:07.971673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:07.971673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:07.971673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:07.971673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:07.973212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:07.973212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:07.973212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:07.973212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:07.973212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:07.973212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:07.973212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:07.973212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:07.973212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:07.973212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:07.973212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:07.973212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:07.973212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:07.973212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:07.973212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:07.973896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:07.973896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:07.973896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:07.973896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:12:07.973896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:07.973896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:07.973896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:07.973896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:07.973896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:07.973896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:07.973896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:07.973896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:07.973896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:07.973896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:07.973896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:07.973896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:07.973896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:07.973896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:07.973896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:07.974657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:07.974657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:07.974657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:07.974657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:07.974657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:07.974657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:08.707877 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:08.708433 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:08.708587 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:08.709325 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:08.709577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.710295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:08.710295 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:08.710295 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:08.710295 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:08.710295 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:08.710295 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:08.710295 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.710295 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:08.711326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.711326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:08.711326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:08.711326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:08.711326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:08.711326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:12:08.711326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:08.711326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:08.711326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:08.711326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:08.711326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:08.711326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:08.711956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:08.711956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:08.711956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:08.711956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:08.711956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:08.711956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:08.711956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:08.711956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:08.711956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:08.711956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:08.711956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:08.711956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:08.711956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:08.713181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.713181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:08.713181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.713181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:08.713181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:08.713181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:08.713181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:08.713181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:08.713181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:08.713181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:08.713181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:08.713181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:08.713181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:08.713181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:08.713181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:08.714104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:08.714104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:08.714104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:08.714104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:12:08.714104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:08.714104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:08.714104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:08.714104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:08.714104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:08.714104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:08.714104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:08.714104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:08.714104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:08.714104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:08.714104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:08.714104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:08.714104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:08.714104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:08.714104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:08.715622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:08.715622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:08.715622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:08.715622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:08.715622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.715622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:08.727942 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:08.728450 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:08.728770 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:08.729105 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:12:08.729570 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:08.729837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.730397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:08.730397 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:08.730397 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:08.730397 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:08.730397 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:08.730397 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:08.730397 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.730397 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:08.731300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.731300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:12:08.731300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:08.731300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:08.731300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:08.731300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:08.731300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:08.731300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:08.731300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:08.731300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:08.731300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:08.731300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:12:08.731300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:08.731300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:12:08.731300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:12:08.732029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:08.732029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:12:08.732029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:08.732029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:12:08.732029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:12:08.732029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:08.732029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:12:08.732029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.732029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:12:08.732956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.732956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:08.732956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:08.732956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:08.732956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:08.732956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:12:08.732956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:12:08.732956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:12:08.732956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:08.732956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:08.732956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:08.732956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:08.734011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:08.734011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:08.734011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:08.734011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:08.734011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:08.734011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:08.734011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:08.734011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:08.734011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:08.734011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:08.734011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:08.734011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:08.734011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:08.734011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:08.734577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:08.734577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:08.734577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.734577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:08.734577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.734577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:08.734577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:08.734577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:08.734577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:08.734577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:08.734577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:08.734577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:08.734577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:08.734577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:08.736173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:08.736173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:08.736173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:08.736173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:08.736173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:08.736173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:08.736173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:12:08.736173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:12:08.736173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:12:08.736173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:08.736173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:08.736173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:08.736173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:08.736173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:08.736173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:08.736173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:08.736173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:08.736173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:08.736173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:08.737513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:08.737513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:08.737513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:08.737513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:08.737513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:08.737513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:08.737513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:08.737513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:08.737513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:08.737513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:08.737513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.737513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:08.739738 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:08.740076 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:08.740194 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:08.740340 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:08.740999 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:08.741323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.742040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:08.742040 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:08.742040 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:08.742040 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:08.742040 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:08.742040 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:08.742040 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.742040 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:08.742912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.742912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:12:08.742912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:08.742912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:08.742912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:08.742912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:08.742912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:08.742912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:08.742912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:08.742912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:08.742912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:08.742912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:12:08.742912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:08.742912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:12:08.742912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:12:08.743605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:08.743605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:12:08.743605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:08.743605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:12:08.743605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:12:08.743605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:08.743605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:12:08.743605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.743605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:12:08.744078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.744078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:08.744078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:08.744078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:08.744078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:08.744078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:12:08.744078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:12:08.744078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:08.744078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:08.744078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:08.744078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:08.745077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:08.745077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:08.745077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:08.745077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:08.745077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:08.745077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:08.745077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:08.745077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:08.745077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:08.745077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:08.745077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:08.745077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:08.745077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:08.745077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:08.746293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:08.746293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:08.746293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.746293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:08.746293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.746293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:08.746293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:08.746293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:08.746293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:08.746293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:08.746293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:08.746293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:08.746293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:08.746293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:08.746952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:08.746952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:08.746952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:08.746952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:08.746952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:08.746952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:08.746952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:12:08.746952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:12:08.746952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:08.746952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:08.746952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:08.746952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:08.746952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:08.746952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:08.746952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:08.746952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:08.746952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:08.746952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:08.746952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:08.747404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:08.747404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:08.747404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:08.747404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:08.747404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:08.747404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:08.747404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:08.747404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:08.747404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:08.747404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.747404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:08.747404 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:08.747404 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:08.747741 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:08.747741 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:08.748335 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:08.748841 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:08.749099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.750004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:08.750004 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:08.750004 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:08.750004 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:08.750004 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:08.750004 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:08.750004 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.750004 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:08.750943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.750943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:08.750943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:08.750943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:08.750943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:08.750943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:12:08.750943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:08.750943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:08.750943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:08.750943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:08.750943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:08.750943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:08.751513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:08.751513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:08.751513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:08.751513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:08.751513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:08.751513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:08.751513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:08.751513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:08.751513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:08.751513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:08.751513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:08.751513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:08.751513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:08.752090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.752090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:08.752090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.752090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:08.752090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:08.752090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:08.752090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:08.752090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:08.752090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:08.752090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:08.752090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:08.752090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:08.752090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:08.752090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:08.752090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:08.752607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:08.752607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:08.752607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:08.752607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:12:08.752607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:08.752607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:08.752607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:08.752607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:08.752607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:08.752607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:08.752607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:08.752607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:08.752607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:08.752607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:08.752607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:08.752607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:08.752607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:08.752607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:08.752607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:08.753133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:08.753133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:08.753133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:08.753133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:08.753133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:08.753133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:12.931455 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:12.932245 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:12.932245 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:12.933184 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:12.968235 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:12.968788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:12.969717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:12.969717 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:12.969717 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:12.969717 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:12.969717 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:12.969717 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:12.969717 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:12.969717 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:12.970823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:12.970823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:12.970823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:12.970823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:12.970823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:12.970823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:12:12.970823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:12.970823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:12.970823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:12.970823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:12.970823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:12.970823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:12.971427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:12.971427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:12.971427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:12.971427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:12.971427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:12.971427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:12.971427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:12.971427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:12.971427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:12.971427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:12.971427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:12.971427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:12.971427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:12.972134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:12.972134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:12.972134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:12.972134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:12.972134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:12.972134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:12.972134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:12.972134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:12.972134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:12.972134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:12.972134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:12.972134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:12.972134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:12.972134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:12.972134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:12.972903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:12.972903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:12.972903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:12.972903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:12:12.972903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:12.972903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:12.972903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:12.972903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:12.972903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:12.972903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:12.972903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:12.972903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:12.972903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:12.972903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:12.972903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:12.972903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:12.972903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:12.972903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:12.972903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:12.974155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:12.974155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:12.974155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:12.974155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:12.974155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:12.974155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:13.708661 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:13.709388 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:13.709585 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:13.710706 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:13.711126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.712040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:13.712040 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:13.712040 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:13.712040 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:13.712040 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:13.712040 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:13.712040 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.712040 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:13.713562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.713562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:13.713562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:13.713562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:13.713562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:13.713562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:12:13.713562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:13.713562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:13.713562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:13.713562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:13.713562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:13.713562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:13.714184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:13.714184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:13.714184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:13.714184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:13.714184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:13.714184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:13.714184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:13.714184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:13.714184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:13.714184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:13.714184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:13.714184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:13.714184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:13.714783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.714783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:13.715341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.715341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:13.715341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:13.715341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:13.715341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:13.715341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:13.715341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:13.715341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:13.715341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:13.715341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:13.715341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:13.715341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:13.715341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:13.715341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:13.715341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:13.715341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:13.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:12:13.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:13.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:13.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:13.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:13.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:13.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:13.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:13.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:13.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:13.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:13.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:13.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:13.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:13.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:13.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:13.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:13.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:13.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:13.716939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:13.716939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.716939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:13.728539 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:13.728947 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:13.729130 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:13.729314 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:12:13.729695 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:13.729949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.730561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:13.730561 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:13.730561 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:13.730561 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:13.730561 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:13.730561 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:13.730561 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.730561 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:13.731818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.731818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:12:13.731818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:13.731818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:13.731818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:13.731818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:13.731818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:13.731818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:13.731818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:13.731818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:13.731818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:13.731818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:12:13.731818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:13.731818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:12:13.731818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:12:13.732450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:13.732450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:12:13.732450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:13.732450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:12:13.732450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:12:13.732450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:13.732450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:12:13.732450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.732450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:12:13.733300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.733300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:13.733300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:13.733300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:13.733300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:13.733300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:12:13.733300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:12:13.733300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:12:13.733300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:13.733300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:13.733300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:13.733300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:13.733918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:13.733918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:13.733918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:13.733918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:13.733918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:13.733918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:13.733918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:13.733918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:13.733918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:13.733918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:13.733918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:13.733918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:13.733918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:13.733918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:13.734725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:13.734725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:13.734725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.734725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:13.734975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.734975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:13.734975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:13.734975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:13.734975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:13.734975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:13.734975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:13.734975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:13.734975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:13.734975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:13.734975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:13.734975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:13.734975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:13.734975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:13.734975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:13.734975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:13.735799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:12:13.735799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:12:13.735799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:12:13.735799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:13.735799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:13.735799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:13.735799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:13.735799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:13.735799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:13.735799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:13.735799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:13.735799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:13.735799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:13.735799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:13.735799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:13.735799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:13.735799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:13.735799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:13.736577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:13.736577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:13.736577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:13.736577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:13.736577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:13.736577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.736577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:13.740165 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:13.740527 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:13.740702 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:13.740890 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:13.741372 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:13.741809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.742698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:13.742698 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:13.742698 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:13.742698 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:13.742698 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:13.742698 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:13.742698 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.742698 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:13.743468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.743468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:12:13.743468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:13.743468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:13.743468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:13.743468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:13.743468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:13.743468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:13.743468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:13.743468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:13.743468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:13.743468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:12:13.743468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:13.743468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:12:13.743468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:12:13.743916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:13.743916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:12:13.743916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:13.743916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:12:13.743916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:12:13.743916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:13.743916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:12:13.743916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.743916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:12:13.744421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.744421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:13.744421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:13.744421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:13.744421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:13.744421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:12:13.744421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:12:13.744421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:13.744421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:13.744421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:13.744421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:13.744810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:13.744810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:13.744810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:13.744810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:13.744810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:13.744810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:13.744810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:13.744810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:13.744810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:13.744810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:13.744810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:13.744810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:13.744810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:13.744810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:13.745254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:13.745254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:13.745254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.745254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:13.745553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.745553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:13.745553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:13.745553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:13.745553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:13.745553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:13.745553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:13.745553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:13.745553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:13.745553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:13.745553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:13.745553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:13.745553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:13.745553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:13.745553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:13.745553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:13.745553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:12:13.746163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:12:13.746163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:13.746163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:13.746163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:13.746163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:13.746163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:13.746163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:13.746163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:13.746163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:13.746163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:13.746163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:13.746163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:13.746163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:13.746163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:13.746163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:13.746163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:13.746163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:13.746163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:13.746163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:13.746706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:13.746706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:13.746706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.746706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:13.747314 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:13.747568 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:13.747723 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:13.747929 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:13.748399 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:13.748811 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:13.749003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.749737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:13.749737 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:13.749737 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:13.749737 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:13.749737 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:13.749737 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:13.749737 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.749737 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:13.750636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.750636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:13.750636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:13.750636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:13.750636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:13.750636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:12:13.750636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:13.750636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:13.750636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:13.750636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:13.750636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:13.750636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:13.751124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:13.751124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:13.751124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:13.751124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:13.751124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:13.751124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:13.751124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:13.751124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:13.751124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:13.751124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:13.751124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:13.751124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:13.751124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:13.751600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.751600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:13.751699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.751699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:13.751699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:13.751699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:13.751699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:13.751699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:13.751699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:13.751699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:13.751699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:13.751699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:13.751699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:13.751699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:13.751699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:13.751699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:13.751699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:13.751699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:13.752243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:12:13.752243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:13.752243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:13.752243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:13.752243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:13.752243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:13.752243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:13.752243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:13.752243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:13.752243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:13.752243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:13.752243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:13.752243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:13.752243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:13.752243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:13.752243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:13.752243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:13.752243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:13.752243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:13.752830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:13.752830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:13.752830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:17.932041 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:17.932849 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:17.932849 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:17.933942 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:17.959866 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:17.960254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:17.961091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:17.961091 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:17.961091 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:17.961091 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:17.961091 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:17.961091 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:17.961091 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:17.961091 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:17.961871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:17.961871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:17.961871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:17.961871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:17.961871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:17.961871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:12:17.961871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:17.961871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:17.961871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:17.961871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:17.961871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:17.961871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:17.962457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:17.962457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:17.962457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:17.962457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:17.962457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:17.962457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:17.962457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:17.962457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:17.962457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:17.962457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:17.962457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:17.962457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:17.962457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:17.963218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:17.963218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:17.963218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:17.963218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:17.963218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:17.963218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:17.963218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:17.963218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:17.963218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:17.963218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:17.963218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:17.963218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:17.963218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:17.963218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:17.963218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:17.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:17.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:17.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:17.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:12:17.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:17.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:17.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:17.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:17.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:17.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:17.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:17.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:17.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:17.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:17.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:17.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:17.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:17.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:17.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:17.964856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:17.964856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:17.964856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:17.964856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:17.964856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:17.964856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:18.709417 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:18.709884 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:18.710085 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:18.711093 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:18.711402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.712311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:18.712311 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:18.712311 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:18.712311 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:18.712311 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:18.712311 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:18.712311 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.712311 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:18.714178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.714178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:18.714178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:18.714178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:18.714178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:18.714178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:12:18.714178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:18.714178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:18.714178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:18.714178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:18.714178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:18.714178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:18.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:18.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:18.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:18.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:18.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:18.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:18.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:18.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:18.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:18.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:18.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:18.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:18.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:18.714986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.714986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:18.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:18.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:18.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:18.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:18.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:18.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:18.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:18.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:18.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:18.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:18.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:18.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:18.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:18.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:18.716048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:18.716554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:12:18.716554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:18.716554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:18.716554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:18.716554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:18.716554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:18.716554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:18.716554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:18.716554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:18.716554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:18.716554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:18.716554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:18.716554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:18.716554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:18.716554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:18.716554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:18.716554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:18.716554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:18.716554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:18.717145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:18.717145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.717145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:18.728988 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:18.729233 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:18.729338 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:18.729466 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:12:18.730026 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:18.730334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.731053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:18.731053 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:18.731053 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:18.731053 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:18.731053 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:18.731053 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:18.731053 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.731053 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:18.731966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.731966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:12:18.731966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:18.731966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:18.731966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:18.731966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:18.731966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:18.731966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:18.731966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:18.731966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:18.731966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:18.731966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:12:18.731966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:18.731966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:12:18.731966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:12:18.732830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:18.732830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:12:18.732830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:18.732830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:12:18.732830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:12:18.732830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:18.732830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:12:18.732830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.732830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:12:18.733267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.733267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:18.733267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:18.733267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:18.733267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:18.733267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:12:18.733267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:12:18.733267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:12:18.733267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:18.733267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:18.733267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:18.733267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:18.733982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:18.733982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:18.733982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:18.733982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:18.733982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:18.733982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:18.733982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:18.733982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:18.733982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:18.733982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:18.733982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:18.733982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:18.733982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:18.733982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:18.734757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:18.734757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:18.734757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.734757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:18.735039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.735039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:18.735039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:18.735039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:18.735039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:18.735039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:18.735039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:18.735039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:18.735039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:18.735039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:18.735039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:18.735039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:18.735039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:18.735039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:18.735039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:18.735039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:18.735786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:12:18.735786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:12:18.735786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:12:18.735786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:18.735786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:18.735786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:18.735786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:18.735786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:18.735786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:18.735786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:18.735786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:18.735786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:18.735786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:18.735786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:18.735786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:18.735786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:18.735786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:18.735786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:18.736831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:18.736831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:18.736831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:18.736831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:18.736831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:18.736831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.736831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:18.740499 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:18.740730 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:18.740843 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:18.740979 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:18.741420 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:18.741830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.742487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:18.742487 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:18.742487 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:18.742487 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:18.742487 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:18.742487 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:18.742487 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.742487 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:18.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:12:18.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:18.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:18.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:18.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:18.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:18.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:18.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:18.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:18.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:18.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:12:18.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:18.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:12:18.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:12:18.743584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:18.743584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:12:18.743584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:18.743584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:12:18.743584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:12:18.743584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:18.743584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:12:18.743584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.743584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:12:18.744197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.744197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:18.744197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:18.744197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:18.744197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:18.744197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:12:18.744197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:12:18.744197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:18.744197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:18.744197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:18.744197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:18.744542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:18.744542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:18.744542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:18.744542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:18.744542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:18.744542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:18.744542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:18.744542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:18.744542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:18.744542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:18.744542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:18.744542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:18.744542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:18.744542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:18.744964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:18.744964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:18.744964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.744964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:18.745906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.745906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:18.745906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:18.745906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:18.745906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:18.745906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:18.745906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:18.745906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:18.745906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:18.745906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:18.745906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:18.745906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:18.745906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:18.745906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:18.745906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:18.745906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:18.745906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:12:18.746434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:12:18.746434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:18.746434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:18.746434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:18.746434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:18.746434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:18.746434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:18.746434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:18.746434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:18.746434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:18.746434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:18.746434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:18.746434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:18.746434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:18.746434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:18.746434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:18.746434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:18.746434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:18.746434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:18.746995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:18.746995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:18.746995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.746995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:18.747748 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:18.748049 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:18.748221 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:18.748467 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:18.748897 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:18.749226 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:18.749408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.750085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:18.750085 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:18.750085 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:18.750085 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:18.750085 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:18.750085 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:18.750085 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.750085 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:18.751349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.751349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:18.751349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:18.751349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:18.751349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:18.751349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:12:18.751349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:18.751349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:18.751349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:18.751349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:18.751349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:18.751349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:18.751802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:18.751802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:18.751802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:18.751802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:18.751802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:18.751802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:18.751802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:18.751802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:18.751802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:18.751802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:18.751802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:18.751802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:18.751802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:18.752162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.752162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:18.752668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.752668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:18.752668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:18.752668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:18.752668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:18.752668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:18.752668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:18.752668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:18.752668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:18.752668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:18.752668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:18.752668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:18.752668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:18.752668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:18.752668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:18.752668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:18.753161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:12:18.753161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:18.753161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:18.753161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:18.753161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:18.753161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:18.753161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:18.753161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:18.753161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:18.753161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:18.753161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:18.753161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:18.753161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:18.753161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:18.753161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:18.753161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:18.753161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:18.753161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:18.753161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:18.753724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:18.753724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:18.753724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:22.932638 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:22.933743 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:22.933743 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:22.934880 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:22.966732 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:22.967041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:22.967589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:22.967589 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:22.967589 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:22.967589 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:22.967589 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:22.967589 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:22.967589 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:22.967589 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:22.968314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:22.968314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:22.968314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:22.968314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:22.968314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:22.968314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:12:22.968314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:22.968314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:22.968314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:22.968314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:22.968314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:22.968314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:22.968930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:22.968930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:22.968930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:22.968930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:22.968930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:22.968930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:22.968930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:22.968930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:22.968930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:22.968930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:22.968930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:22.968930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:22.968930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:22.969517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:22.969517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:22.969517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:22.969517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:22.969517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:22.969517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:22.969517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:22.969517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:22.969517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:22.969517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:22.969517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:22.969517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:22.969517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:22.969517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:22.969517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:22.970166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:22.970166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:22.970166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:22.970166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:12:22.970166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:22.970166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:22.970166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:22.970166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:22.970166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:22.970166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:22.970166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:22.970166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:22.970166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:22.970166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:22.970166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:22.970166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:22.970166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:22.970166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:22.970166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:22.970933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:22.970933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:22.970933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:22.970933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:22.970933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:22.970933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:23.710080 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:23.710664 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:23.710864 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:23.711692 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:23.712055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.712885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:23.712885 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:23.712885 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:23.712885 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:23.712885 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:23.712885 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:23.712885 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.712885 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:23.713859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.713859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:23.713859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:23.713859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:23.713859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:23.713859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:12:23.713859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:23.713859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:23.713859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:23.713859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:23.713859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:23.713859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:23.714416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:23.714416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:23.714416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:23.714416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:23.714416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:23.714416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:23.714416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:23.714416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:23.714416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:23.714416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:23.714416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:23.714416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:23.714416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:23.714947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.714947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:23.714947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.714947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:23.714947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:23.714947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:23.714947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:23.714947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:23.714947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:23.714947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:23.714947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:23.714947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:23.714947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:23.714947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:23.714947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:23.715567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:23.715567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:23.715567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:23.715567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:12:23.715567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:23.715567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:23.715567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:23.715567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:23.715567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:23.715567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:23.715567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:23.715567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:23.715567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:23.715567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:23.715567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:23.715567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:23.715567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:23.715567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:23.715567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:23.716317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:23.716317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:23.716317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:23.716317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:23.716317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.716317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:23.729420 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:23.729797 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:23.729931 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:23.730039 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:12:23.730489 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:23.730834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.731551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:23.731551 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:23.731551 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:23.731551 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:23.731551 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:23.731551 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:23.731551 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.731551 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:23.732816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.732816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:12:23.732816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:23.732816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:23.732816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:23.732816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:23.732816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:23.732816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:23.732816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:23.732816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:23.732816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:23.732816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:12:23.732816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:23.732816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:12:23.732816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:12:23.733547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:23.733547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:12:23.733547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:23.733547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:12:23.733547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:12:23.733547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:23.733547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:12:23.733547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.733547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:12:23.734454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.734454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:23.734454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:23.734454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:23.734454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:23.734454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:12:23.734454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:12:23.734454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:12:23.734454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:23.734454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:23.734454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:23.734454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:23.735766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:23.735766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:23.735766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:23.735766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:23.735766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:23.735766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:23.735766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:23.735766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:23.735766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:23.735766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:23.735766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:23.735766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:23.735766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:23.735766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:23.737313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:23.737313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:23.737313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.737313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:23.737313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.737313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:23.737313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:23.737313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:23.737313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:23.737313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:23.737313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:23.737313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:23.737313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:23.737313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:23.738709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:23.738709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:23.738709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:23.738709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:23.738709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:23.738709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:23.738709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:12:23.738709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:12:23.738709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:12:23.738709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:23.738709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:23.738709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:23.738709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:23.738709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:23.738709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:23.738709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:23.738709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:23.738709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:23.738709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:23.739286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:23.739286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:23.739286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:23.739286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:23.739286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:23.739286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:23.739286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:23.739286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:23.739286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:23.739286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:23.739286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.739286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:23.740859 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:23.741127 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:23.741232 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:23.741364 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:23.741695 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:23.741930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.742362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:23.742362 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:23.742362 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:23.742362 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:23.742362 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:23.742362 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:23.742362 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.742362 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:23.743069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.743069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:12:23.743069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:23.743069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:23.743069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:23.743069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:23.743069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:23.743069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:23.743069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:23.743069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:23.743069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:23.743069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:12:23.743069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:23.743069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:12:23.743069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:12:23.743456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:23.743456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:12:23.743456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:23.743456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:12:23.743456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:12:23.743456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:23.743456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:12:23.743456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.743456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:12:23.743954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.743954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:23.743954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:23.743954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:23.743954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:23.743954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:12:23.743954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:12:23.743954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:23.743954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:23.743954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:23.743954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:23.744670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:23.744670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:23.744670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:23.744670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:23.744670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:23.744670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:23.744670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:23.744670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:23.744670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:23.744670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:23.744670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:23.744670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:23.744670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:23.744670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:23.745412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:23.745412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:23.745412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.745412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:23.745412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.745412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:23.745412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:23.745412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:23.745412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:23.745412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:23.745412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:23.745412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:23.745412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:23.745412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:23.745842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:23.745842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:23.745842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:23.745842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:23.745842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:23.745842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:23.745842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:12:23.745842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:12:23.745842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:23.745842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:23.745842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:23.745842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:23.745842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:23.745842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:23.745842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:23.745842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:23.745842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:23.745842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:23.745842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:23.746285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:23.746285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:23.746285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:23.746285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:23.746285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:23.746285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:23.746285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:23.746285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:23.746285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:23.746285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.746285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:23.748047 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:23.748309 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:23.748414 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:23.748569 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:23.748989 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:23.749355 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:23.749598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.750267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:23.750267 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:23.750267 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:23.750267 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:23.750267 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:23.750267 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:23.750267 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.750267 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:23.751067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.751067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:23.751067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:23.751067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:23.751067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:23.751067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:12:23.751067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:23.751067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:23.751067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:23.751067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:23.751067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:23.751067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:23.751386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:23.751386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:23.751386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:23.751386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:23.751386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:23.751386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:23.751386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:23.751386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:23.751386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:23.751386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:23.751386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:23.751386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:23.751386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:23.751714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.751714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:23.752022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.752022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:23.752022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:23.752022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:23.752022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:23.752022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:23.752022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:23.752022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:23.752022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:23.752022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:23.752022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:23.752022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:23.752022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:23.752022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:23.752022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:23.752022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:23.752439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:12:23.752439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:23.752439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:23.752439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:23.752439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:23.752439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:23.752439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:23.752439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:23.752439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:23.752439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:23.752439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:23.752439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:23.752439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:23.752439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:23.752439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:23.752439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:23.752439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:23.752439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:23.752439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:23.752916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:23.752916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:23.752916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:27.933293 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:27.934442 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:27.934442 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:27.935786 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:27.971072 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:27.971392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:27.972107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:27.972107 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:27.972107 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:27.972107 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:27.972107 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:27.972107 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:27.972107 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:27.972107 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:27.972909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:27.972909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:27.972909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:27.972909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:27.972909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:27.972909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:12:27.972909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:27.972909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:27.972909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:27.972909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:27.972909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:27.972909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:27.973301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:27.973301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:27.973301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:27.973301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:27.973301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:27.973301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:27.973301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:27.973301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:27.973301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:27.973301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:27.973301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:27.973301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:27.973301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:27.973835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:27.973835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:27.973835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:27.973835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:27.973835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:27.973835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:27.973835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:27.973835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:27.973835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:27.973835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:27.973835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:27.973835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:27.973835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:27.973835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:27.973835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:27.974235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:27.974235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:27.974235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:27.974235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:12:27.974235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:27.974235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:27.974235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:27.974235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:27.974235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:27.974235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:27.974235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:27.974235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:27.974235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:27.974235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:27.974235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:27.974235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:27.974235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:27.974235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:27.974235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:27.974726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:27.974726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:27.974726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:27.974726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:27.974726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:27.974726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:28.710931 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:28.711631 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:28.711839 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:28.713094 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:28.713420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.714317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:28.714317 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:28.714317 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:28.714317 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:28.714317 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:28.714317 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:28.714317 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.714317 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:28.715962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.715962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:28.715962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:28.715962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:28.715962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:28.715962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:12:28.715962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:28.715962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:28.715962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:28.715962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:28.715962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:28.715962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:28.716697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:28.716697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:28.716697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:28.716697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:28.716697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:28.716697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:28.716697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:28.716697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:28.716697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:28.716697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:28.716697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:28.716697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:28.716697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:28.717290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.717290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:28.717897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.717897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:28.717897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:28.717897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:28.717897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:28.717897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:28.717897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:28.717897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:28.717897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:28.717897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:28.717897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:28.717897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:28.717897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:28.717897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:28.717897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:28.717897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:28.718615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:12:28.718615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:28.718615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:28.718615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:28.718615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:28.718615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:28.718615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:28.718615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:28.718615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:28.718615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:28.718615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:28.718615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:28.718615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:28.718615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:28.718615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:28.718615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:28.718615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:28.718615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:28.718615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:28.719270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:28.719270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.719270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:28.730017 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:28.730334 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:28.730546 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:28.730776 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:12:28.731356 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:28.731729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.732574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:28.732574 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:28.732574 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:28.732574 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:28.732574 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:28.732574 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:28.732574 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.732574 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:28.733979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.733979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:12:28.733979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:28.733979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:28.733979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:28.733979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:28.733979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:28.733979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:28.733979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:28.733979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:28.733979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:28.733979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:12:28.733979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:28.733979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:12:28.733979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:12:28.734776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:28.734776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:12:28.734776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:28.734776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:12:28.734776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:12:28.734776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:28.734776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:12:28.734776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.734776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:12:28.735583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.735583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:28.735583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:28.735583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:28.735583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:28.735583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:12:28.735583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:12:28.735583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:12:28.735583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:28.735583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:28.735583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:28.735583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:28.736248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:28.736248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:28.736248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:28.736248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:28.736248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:28.736248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:28.736248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:28.736248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:28.736248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:28.736248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:28.736248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:28.736248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:28.736248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:28.736248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:28.736925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:28.736925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:28.736925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.736925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:28.737131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.737131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:28.737131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:28.737131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:28.737131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:28.737131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:28.737131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:28.737131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:28.737131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:28.737131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:28.737131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:28.737131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:28.737131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:28.737131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:28.737131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:28.737131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:28.737850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:12:28.737850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:12:28.737850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:12:28.737850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:28.737850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:28.737850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:28.737850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:28.737850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:28.737850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:28.737850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:28.737850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:28.737850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:28.737850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:28.737850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:28.737850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:28.737850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:28.737850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:28.737850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:28.738791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:28.738791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:28.738791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:28.738791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:28.738791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:28.738791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.738791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:28.741298 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:28.741604 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:28.741797 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:28.741934 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:28.742403 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:28.742775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.743522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:28.743522 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:28.743522 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:28.743522 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:28.743522 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:28.743522 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:28.743522 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.743522 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:28.744446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.744446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:12:28.744446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:28.744446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:28.744446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:28.744446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:28.744446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:28.744446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:28.744446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:28.744446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:28.744446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:28.744446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:12:28.744446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:28.744446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:12:28.744446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:12:28.744899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:28.744899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:12:28.744899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:28.744899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:12:28.744899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:12:28.744899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:28.744899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:12:28.744899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.744899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:12:28.745943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.745943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:28.745943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:28.745943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:28.745943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:28.745943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:12:28.745943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:12:28.745943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:28.745943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:28.745943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:28.745943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:28.746314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:28.746314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:28.746314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:28.746314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:28.746314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:28.746314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:28.746314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:28.746314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:28.746314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:28.746314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:28.746314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:28.746314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:28.746314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:28.746314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:28.746719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:28.746719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:28.746719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.746719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:28.747935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.747935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:28.747935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:28.747935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:28.747935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:28.747935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:28.747935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:28.747935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:28.747935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:28.747935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:28.747935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:28.747935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:28.747935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:28.747935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:28.747935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:28.747935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:28.747935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:12:28.748429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:12:28.748429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:28.748429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:28.748429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:28.748429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:28.748429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:28.748429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:28.748429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:28.748429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:28.748429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:28.748429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:28.748429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:28.748429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:28.748429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:28.748429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:28.748429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:28.748429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:28.748429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:28.748429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:28.748984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:28.748984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:28.748984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.748984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:28.748984 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:28.748984 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:28.748984 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:28.748984 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:28.749449 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:28.749903 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:28.750097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.750774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:28.750774 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:28.750774 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:28.750774 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:28.750774 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:28.750774 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:28.750774 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.750774 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:28.751688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.751688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:28.751688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:28.751688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:28.751688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:28.751688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:12:28.751688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:28.751688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:28.751688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:28.751688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:28.751688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:28.751688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:28.752101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:28.752101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:28.752101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:28.752101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:28.752101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:28.752101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:28.752101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:28.752101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:28.752101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:28.752101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:28.752101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:28.752101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:28.752101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:28.752471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.752471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:28.752772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.752772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:28.752772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:28.752772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:28.752772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:28.752772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:28.752772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:28.752772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:28.752772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:28.752772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:28.752772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:28.752772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:28.752772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:28.752772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:28.752772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:28.752772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:28.753291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:12:28.753291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:28.753291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:28.753291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:28.753291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:28.753291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:28.753291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:28.753291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:28.753291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:28.753291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:28.753291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:28.753291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:28.753291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:28.753291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:28.753291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:28.753291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:28.753291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:28.753291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:28.753291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:28.753871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:28.753871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:28.753871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:32.933840 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:32.934348 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:32.934447 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:32.935806 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:32.965098 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:32.965279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:32.965819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:32.965819 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:32.965819 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:32.965819 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:32.965819 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:32.965819 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:32.965819 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:32.965819 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:32.966615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:32.966615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:32.966615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:32.966615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:32.966615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:32.966615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:12:32.966615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:32.966615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:32.966615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:32.966615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:32.966615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:32.966615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:32.967053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:32.967053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:32.967053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:32.967053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:32.967053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:32.967053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:32.967053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:32.967053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:32.967053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:32.967053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:32.967053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:32.967053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:32.967053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:32.967458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:32.967458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:32.967597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:32.967597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:32.967597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:32.967597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:32.967597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:32.967597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:32.967597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:32.967597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:32.967597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:32.967597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:32.967597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:32.967597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:32.967597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:32.967597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:32.967597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:32.967597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:32.968133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:12:32.968133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:32.968133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:32.968133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:32.968133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:32.968133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:32.968133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:32.968133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:32.968133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:32.968133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:32.968133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:32.968133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:32.968133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:32.968133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:32.968133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:32.968133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:32.968133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:32.968133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:32.968133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:32.968759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:32.968759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:32.968759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:33.711594 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:33.712063 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:33.712273 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:33.712916 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:33.713092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.713738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:33.713738 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:33.713738 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:33.713738 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:33.713738 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:33.713738 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:33.713738 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.713738 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:33.715221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.715221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:33.715221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:33.715221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:33.715221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:33.715221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:12:33.715221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:33.715221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:33.715221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:33.715221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:33.715221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:33.715221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:33.715621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:33.715621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:33.715621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:33.715621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:33.715621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:33.715621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:33.715621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:33.715621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:33.715621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:33.715621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:33.715621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:33.715621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:33.715621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:33.716007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.716007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:33.716951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.716951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:33.716951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:33.716951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:33.716951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:33.716951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:33.716951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:33.716951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:33.716951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:33.716951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:33.716951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:33.716951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:33.716951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:33.716951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:33.716951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:33.716951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:33.717403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:12:33.717403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:33.717403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:33.717403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:33.717403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:33.717403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:33.717403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:33.717403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:33.717403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:33.717403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:33.717403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:33.717403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:33.717403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:33.717403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:33.717403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:33.717403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:33.717403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:33.717403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:33.717403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:33.718576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:33.718576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.718576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:33.730365 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:33.730716 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:33.731009 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:33.731272 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:12:33.731495 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:33.731659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.732173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:33.732173 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:33.732173 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:33.732173 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:33.732173 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:33.732173 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:33.732173 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.732173 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:33.733583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.733583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:12:33.733583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:33.733583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:33.733583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:33.733583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:33.733583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:33.733583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:33.733583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:33.733583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:33.733583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:33.733583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:12:33.733583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:33.733583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:12:33.733583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:12:33.734523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:33.734523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:12:33.734523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:33.734523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:12:33.734523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:12:33.734523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:33.734523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:12:33.734523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.734523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:12:33.735135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.735135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:33.735135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:33.735135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:33.735135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:33.735135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:12:33.735135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:12:33.735135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:12:33.735135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:33.735135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:33.735135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:33.735135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:33.735832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:33.735832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:33.735832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:33.735832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:33.735832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:33.735832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:33.735832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:33.735832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:33.735832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:33.735832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:33.735832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:33.735832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:33.735832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:33.735832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:33.736433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:33.736433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:33.736433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.736433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:33.737170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.737170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:33.737170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:33.737170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:33.737170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:33.737170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:33.737170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:33.737170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:33.737170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:33.737170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:33.737170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:33.737170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:33.737170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:33.737170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:33.737170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:33.737170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:33.738166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:12:33.738166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:12:33.738166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:12:33.738166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:33.738166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:33.738166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:33.738166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:33.738166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:33.738166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:33.738166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:33.738166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:33.738166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:33.738166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:33.738166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:33.738166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:33.738166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:33.738166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:33.738166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:33.738921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:33.738921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:33.738921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:33.738921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:33.738921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:33.738921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.738921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:33.741876 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:33.742295 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:33.742523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:33.742585 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:33.742917 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:33.743200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.743795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:33.743795 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:33.743795 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:33.743795 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:33.743795 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:33.743795 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:33.743795 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.743795 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:33.744843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.744843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:12:33.744843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:33.744843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:33.744843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:33.744843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:33.744843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:33.744843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:33.744843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:33.744843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:33.744843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:33.744843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:12:33.744843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:33.744843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:12:33.744843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:12:33.745267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:33.745267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:12:33.745267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:33.745267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:12:33.745267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:12:33.745267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:33.745267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:12:33.745267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.745267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:12:33.745924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.745924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:33.745924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:33.745924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:33.745924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:33.745924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:12:33.745924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:12:33.745924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:33.745924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:33.745924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:33.745924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:33.746305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:33.746305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:33.746305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:33.746305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:33.746305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:33.746305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:33.746305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:33.746305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:33.746305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:33.746305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:33.746305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:33.746305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:33.746305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:33.746305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:33.746702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:33.746702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:33.746702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.746702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:33.747095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.747095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:33.747095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:33.747095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:33.747095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:33.747095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:33.747095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:33.747095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:33.747095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:33.747095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:33.747095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:33.747095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:33.747095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:33.747095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:33.747095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:33.747095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:33.747095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:12:33.748281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:12:33.748281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:33.748281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:33.748281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:33.748281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:33.748281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:33.748281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:33.748281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:33.748281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:33.748281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:33.748281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:33.748281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:33.748281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:33.748281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:33.748281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:33.748281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:33.748281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:33.748281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:33.748281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:33.749256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:33.749256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:33.749256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.749256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:33.749256 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:33.749256 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:33.749256 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:33.749256 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:33.750110 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:33.750532 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:33.750664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.751285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:33.751285 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:33.751285 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:33.751285 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:33.751285 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:33.751285 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:33.751285 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.751285 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:33.752213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.752213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:33.752213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:33.752213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:33.752213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:33.752213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:12:33.752213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:33.752213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:33.752213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:33.752213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:33.752213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:33.752213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:33.752569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:33.752569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:33.752569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:33.752569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:33.752569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:33.752569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:33.752569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:33.752569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:33.752569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:33.752569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:33.752569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:33.752569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:33.752569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:33.752965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.752965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:33.753262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.753262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:33.753262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:33.753262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:33.753262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:33.753262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:33.753262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:33.753262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:33.753262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:33.753262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:33.753262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:33.753262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:33.753262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:33.753262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:33.753262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:33.753262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:33.753718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:12:33.753718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:33.753718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:33.753718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:33.753718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:33.753718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:33.753718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:33.753718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:33.753718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:33.753718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:33.753718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:33.753718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:33.753718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:33.753718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:33.753718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:33.753718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:33.753718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:33.753718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:33.753718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:33.754291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:33.754291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:33.754291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:37.934459 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:37.935321 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:37.935321 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:37.936291 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:37.965938 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:37.966209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:37.966827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:37.966827 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:37.966827 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:37.966827 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:37.966827 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:37.966827 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:37.966827 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:37.966827 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:37.967764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:37.967764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:37.967764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:37.967764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:37.967764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:37.967764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:12:37.967764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:37.967764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:37.967764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:37.967764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:37.967764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:37.967764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:37.968115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:37.968115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:37.968115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:37.968115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:37.968115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:37.968115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:37.968115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:37.968115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:37.968115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:37.968115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:37.968115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:37.968115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:37.968115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:37.968453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:37.968453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:37.968811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:37.968811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:37.968811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:37.968811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:37.968811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:37.968811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:37.968811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:37.968811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:37.968811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:37.968811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:37.968811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:37.968811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:37.968811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:37.968811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:37.968811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:37.968811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:37.969252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:12:37.969252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:37.969252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:37.969252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:37.969252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:37.969252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:37.969252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:37.969252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:37.969252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:37.969252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:37.969252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:37.969252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:37.969252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:37.969252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:37.969252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:37.969252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:37.969252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:37.969252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:37.969252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:37.969770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:37.969770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:37.969770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:38.712300 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:38.712984 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:38.713188 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:38.714201 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:38.714502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.715393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:38.715393 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:38.715393 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:38.715393 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:38.715393 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:38.715393 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:38.715393 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.715393 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:38.717001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.717001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:38.717001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:38.717001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:38.717001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:38.717001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:12:38.717001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:38.717001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:38.717001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:38.717001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:38.717001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:38.717001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:38.717573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:38.717573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:38.717573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:38.717573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:38.717573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:38.717573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:38.717573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:38.717573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:38.717573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:38.717573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:38.717573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:38.717573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:38.717573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:38.718161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.718161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:38.718663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.718663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:38.718663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:38.718663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:38.718663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:38.718663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:38.718663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:38.718663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:38.718663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:38.718663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:38.718663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:38.718663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:38.718663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:38.718663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:38.718663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:38.718663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:38.719384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:12:38.719384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:38.719384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:38.719384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:38.719384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:38.719384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:38.719384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:38.719384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:38.719384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:38.719384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:38.719384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:38.719384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:38.719384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:38.719384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:38.719384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:38.719384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:38.719384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:38.719384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:38.719384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:38.720361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:38.720361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.720361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:38.730759 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:38.731139 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:38.731315 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:38.731490 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:12:38.732198 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:38.732510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.733434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:38.733434 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:38.733434 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:38.733434 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:38.733434 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:38.733434 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:38.733434 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.733434 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:38.734821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.734821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:12:38.734821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:38.734821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:38.734821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:38.734821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:38.734821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:38.734821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:38.734821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:38.734821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:38.734821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:38.734821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:12:38.734821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:38.734821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:12:38.734821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:12:38.735854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:38.735854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:12:38.735854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:38.735854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:12:38.735854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:12:38.735854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:38.735854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:12:38.735854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.735854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:12:38.736552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.736552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:38.736552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:38.736552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:38.736552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:38.736552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:12:38.736552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:12:38.736552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:12:38.736552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:38.736552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:38.736552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:38.736552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:38.737232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:38.737232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:38.737232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:38.737232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:38.737232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:38.737232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:38.737232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:38.737232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:38.737232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:38.737232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:38.737232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:38.737232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:38.737232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:38.737232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:38.738103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:38.738103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:38.738103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.738103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:38.738103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.738103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:38.738103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:38.738103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:38.738103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:38.738103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:38.738103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:38.738103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:38.738103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:38.738103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:38.738825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:38.738825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:38.738825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:38.738825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:38.738825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:38.738825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:38.738825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:12:38.738825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:12:38.738825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:12:38.738825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:38.738825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:38.738825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:38.738825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:38.738825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:38.738825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:38.738825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:38.738825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:38.738825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:38.738825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:38.739871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:38.739871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:38.739871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:38.739871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:38.739871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:38.739871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:38.739871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:38.739871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:38.739871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:38.739871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:38.739871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.739871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:38.742327 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:38.742585 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:38.742727 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:38.742853 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:38.743355 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:38.743766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.744466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:38.744466 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:38.744466 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:38.744466 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:38.744466 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:38.744466 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:38.744466 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.744466 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:38.745255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.745255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:12:38.745255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:38.745255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:38.745255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:38.745255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:38.745255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:38.745255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:38.745255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:38.745255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:38.745255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:38.745255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:12:38.745255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:38.745255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:12:38.745255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:12:38.746096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:38.746096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:12:38.746096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:38.746096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:12:38.746096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:12:38.746096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:38.746096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:12:38.746096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.746096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:12:38.746096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.746096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:38.746096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:38.746096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:38.746096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:38.746821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:12:38.746821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:12:38.746821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:38.746821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:38.746821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:38.746821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:38.746821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:38.746821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:38.746821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:38.746821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:38.746821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:38.746821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:38.746821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:38.747499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:38.747499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:38.747499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:38.747499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:38.747499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:38.747499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:38.747499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:38.747499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:38.747499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:38.747499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.747499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:38.747499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.748202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:38.748202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:38.748202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:38.748202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:38.748202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:38.748202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:38.748202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:38.748202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:38.748202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:38.748202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:38.748202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:38.748202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:38.748202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:38.748202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:38.748202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:38.748202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:12:38.748202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:12:38.748202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:38.748202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:38.748932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:38.748932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:38.748932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:38.748932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:38.748932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:38.748932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:38.748932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:38.748932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:38.748932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:38.748932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:38.748932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:38.748932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:38.748932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:38.748932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:38.748932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:38.748932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:38.748932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:38.748932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:38.748932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.748932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:38.749476 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:38.749476 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:38.749572 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:38.749762 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:38.750207 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:38.750526 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:38.750811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.751630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:38.751630 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:38.751630 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:38.751630 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:38.751630 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:38.751630 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:38.751630 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.751630 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:38.752395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.752395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:38.752395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:38.752395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:38.752395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:38.752395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:12:38.752395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:38.752395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:38.752395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:38.752395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:38.752395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:38.752395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:38.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:38.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:38.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:38.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:38.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:38.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:38.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:38.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:38.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:38.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:38.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:38.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:38.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:38.753827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.753827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:38.753827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.753827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:38.753827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:38.753827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:38.753827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:38.753827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:38.753827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:38.753827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:38.753827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:38.753827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:38.753827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:38.753827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:38.753827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:38.754878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:38.754878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:38.754878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:38.754878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:12:38.754878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:38.754878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:38.754878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:38.754878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:38.754878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:38.754878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:38.754878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:38.754878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:38.754878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:38.754878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:38.754878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:38.754878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:38.754878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:38.754878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:38.754878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:38.755972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:38.755972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:38.755972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:38.755972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:38.755972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:38.755972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:42.935426 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:42.936391 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:42.936391 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:42.937634 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:42.967476 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:42.967951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:42.968760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:42.968760 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:42.968760 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:42.968760 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:42.968760 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:42.968760 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:42.968760 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:42.968760 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:42.969419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:42.969419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:42.969419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:42.969419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:42.969419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:42.969419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:12:42.969419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:42.969419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:42.969419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:42.969419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:42.969419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:42.969419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:42.970016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:42.970016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:42.970016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:42.970016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:42.970016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:42.970016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:42.970016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:42.970016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:42.970016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:42.970016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:42.970016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:42.970016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:42.970016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:42.970635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:42.970635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:42.970635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:42.970635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:42.970635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:42.970635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:42.970635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:42.970635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:42.970635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:42.970635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:42.970635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:42.970635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:42.970635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:42.970635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:42.970635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:42.971456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:42.971456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:42.971456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:42.971456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:12:42.971456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:42.971456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:42.971456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:42.971456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:42.971456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:42.971456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:42.971456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:42.971456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:42.971456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:42.971456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:42.971456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:42.971456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:42.971456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:42.971456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:42.971456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:42.972650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:42.972650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:42.972650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:42.972650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:42.972650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:42.972650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:43.713122 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:43.714417 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:43.714797 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:43.716049 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:43.716501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.717607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:43.717607 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:43.717607 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:43.717607 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:43.717607 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:43.717607 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:43.717607 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.717607 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:43.719612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.719612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:43.719612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:43.719612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:43.719612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:43.719612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:12:43.719612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:43.719612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:43.719612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:43.719612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:43.719612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:43.719612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:43.720241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:43.720241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:43.720241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:43.720241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:43.720241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:43.720241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:43.720241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:43.720241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:43.720241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:43.720241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:43.720241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:43.720241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:43.720241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:43.720855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.720855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:43.721607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.721607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:43.721607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:43.721607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:43.721607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:43.721607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:43.721607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:43.721607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:43.721607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:43.721607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:43.721607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:43.721607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:43.721607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:43.721607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:43.721607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:43.721607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:43.722155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:12:43.722155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:43.722155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:43.722155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:43.722155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:43.722155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:43.722155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:43.722155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:43.722155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:43.722155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:43.722155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:43.722155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:43.722155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:43.722155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:43.722155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:43.722155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:43.722155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:43.722155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:43.722155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:43.722707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:43.722707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.722707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:43.731300 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:43.731799 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:43.732000 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:43.732215 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:12:43.732693 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:43.733010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.733755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:43.733755 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:43.733755 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:43.733755 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:43.733755 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:43.733755 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:43.733755 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.733755 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:43.735080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.735080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:12:43.735080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:43.735080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:43.735080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:43.735080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:43.735080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:43.735080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:43.735080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:43.735080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:43.735080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:43.735080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:12:43.735080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:43.735080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:12:43.735080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:12:43.736028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:43.736028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:12:43.736028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:43.736028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:12:43.736028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:12:43.736028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:43.736028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:12:43.736028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.736028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:12:43.736889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.736889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:43.736889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:43.736889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:43.736889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:43.736889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:12:43.736889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:12:43.736889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:12:43.736889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:43.736889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:43.736889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:43.736889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:43.737800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:43.737800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:43.737800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:43.737800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:43.737800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:43.737800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:43.737800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:43.737800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:43.737800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:43.737800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:43.737800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:43.737800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:43.737800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:43.737800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:43.738868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:43.738868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:43.738868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.738868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:43.738868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.738868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:43.738868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:43.738868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:43.738868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:43.738868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:43.738868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:43.738868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:43.738868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:43.738868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:43.739418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:43.739418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:43.739418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:43.739418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:43.739418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:43.739418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:43.739418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:12:43.739418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:12:43.739418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:12:43.739418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:43.739418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:43.739418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:43.739418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:43.739418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:43.739418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:43.739418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:43.739418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:43.739418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:43.739418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:43.740789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:43.740789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:43.740789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:43.740789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:43.740789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:43.740789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:43.740789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:43.740789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:43.740789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:43.740789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:43.740789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.740789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:43.742804 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:43.743123 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:43.743275 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:43.743421 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:43.744009 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:43.744448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.745354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:43.745354 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:43.745354 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:43.745354 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:43.745354 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:43.745354 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:43.745354 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.745354 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:43.746228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.746228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:12:43.746228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:43.746228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:43.746228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:43.746228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:43.746228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:43.746228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:43.746228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:43.746228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:43.746228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:43.746228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:12:43.746228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:43.746228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:12:43.746228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:12:43.746741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:43.746741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:12:43.746741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:43.746741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:12:43.746741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:12:43.746741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:43.746741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:12:43.746741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.746741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:12:43.747550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.747550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:43.747550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:43.747550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:43.747550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:43.747550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:12:43.747550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:12:43.747550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:43.747550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:43.747550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:43.747550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:43.747948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:43.747948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:43.747948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:43.747948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:43.747948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:43.747948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:43.747948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:43.747948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:43.747948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:43.747948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:43.747948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:43.747948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:43.747948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:43.747948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:43.748307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:43.748307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:43.748307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.748307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:43.749872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.749872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:43.749872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:43.749872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:43.749872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:43.749872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:43.749872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:43.749872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:43.749872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:43.749872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:43.749872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:43.749872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:43.749872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:43.749872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:43.749872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:43.749872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:43.749872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:12:43.750349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:12:43.750349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:43.750349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:43.750349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:43.750349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:43.750349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:43.750349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:43.750349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:43.750349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:43.750349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:43.750349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:43.750349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:43.750349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:43.750349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:43.750349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:43.750349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:43.750349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:43.750349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:43.750349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:43.750887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:43.750887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:43.750887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.750887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:43.750887 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:43.750887 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:43.751096 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:43.751146 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:43.751695 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:43.752121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:43.752325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.753165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:43.753165 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:43.753165 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:43.753165 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:43.753165 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:43.753165 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:43.753165 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.753165 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:43.754114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.754114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:43.754114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:43.754114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:43.754114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:43.754114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:12:43.754114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:43.754114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:43.754114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:43.754114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:43.754114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:43.754114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:43.754669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:43.754669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:43.754669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:43.754669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:43.754669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:43.754669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:43.754669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:43.754669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:43.754669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:43.754669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:43.754669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:43.754669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:43.754669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:43.755186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.755186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:43.755186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.755186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:43.755186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:43.755186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:43.755186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:43.755186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:43.755186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:43.755186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:43.755186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:43.755186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:43.755186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:43.755186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:43.755186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:43.755757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:43.755757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:43.755757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:43.755757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:12:43.755757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:43.755757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:43.755757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:43.755757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:43.755757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:43.755757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:43.755757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:43.755757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:43.755757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:43.755757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:43.755757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:43.755757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:43.755757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:43.755757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:43.755757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:43.756474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:43.756474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:43.756474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:43.756474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:43.756474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:43.756474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:47.936334 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:47.937179 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:47.937179 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:47.938423 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:47.969506 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:47.970314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:47.971523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:47.971523 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:47.971523 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:47.971523 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:47.971523 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:47.971523 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:47.971523 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:47.971523 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:47.972402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:47.972402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:47.972402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:47.972402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:47.972402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:47.972402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:12:47.972402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:47.972402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:47.972402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:47.972402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:47.972402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:47.972402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:47.973083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:47.973083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:47.973083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:47.973083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:47.973083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:47.973083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:47.973083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:47.973083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:47.973083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:47.973083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:47.973083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:47.973083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:47.973083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:47.973777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:47.973777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:47.973777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:47.973777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:47.973777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:47.973777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:47.973777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:47.973777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:47.973777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:47.973777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:47.973777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:47.973777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:47.973777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:47.973777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:47.973777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:47.974544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:47.974544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:47.974544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:47.974544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:12:47.974544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:47.974544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:47.974544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:47.974544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:47.974544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:47.974544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:47.974544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:47.974544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:47.974544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:47.974544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:47.974544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:47.974544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:47.974544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:47.974544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:47.974544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:47.975561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:47.975561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:47.975561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:47.975561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:47.975561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:47.975561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:48.713814 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:48.714405 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:48.714638 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:48.715632 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:48.716004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.716923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:48.716923 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:48.716923 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:48.716923 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:48.716923 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:48.716923 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:48.716923 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.716923 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:48.718995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.718995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:48.718995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:48.718995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:48.718995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:48.718995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:12:48.718995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:48.718995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:48.718995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:48.718995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:48.718995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:48.718995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:48.719517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:48.719517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:48.719517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:48.719517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:48.719517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:48.719517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:48.719517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:48.719517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:48.719517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:48.719517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:48.719517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:48.719517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:48.719517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:48.720065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.720065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:48.720938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.720938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:48.720938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:48.720938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:48.720938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:48.720938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:48.720938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:48.720938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:48.720938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:48.720938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:48.720938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:48.720938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:48.720938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:48.720938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:48.720938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:48.720938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:48.721533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:12:48.721533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:48.721533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:48.721533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:48.721533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:48.721533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:48.721533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:48.721533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:48.721533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:48.721533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:48.721533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:48.721533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:48.721533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:48.721533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:48.721533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:48.721533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:48.721533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:48.721533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:48.721533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:48.722311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:48.722311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.722311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:48.733650 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:48.734299 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:48.734573 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:48.734803 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:12:48.735484 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:48.735819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.736794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:48.736794 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:48.736794 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:48.736794 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:48.736794 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:48.736794 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:48.736794 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.736794 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:48.738478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.738478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:12:48.738478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:48.738478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:48.738478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:48.738478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:48.738478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:48.738478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:48.738478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:48.738478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:48.738478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:48.738478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:12:48.738478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:48.738478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:12:48.738478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:12:48.739168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:48.739168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:12:48.739168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:48.739168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:12:48.739168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:12:48.739168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:48.739168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:12:48.739168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.739168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:12:48.739690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.739690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:48.739690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:48.739690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:48.739690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:48.739690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:12:48.739690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:12:48.739690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:12:48.739690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:48.739690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:48.739690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:48.739690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:48.740239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:48.740239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:48.740239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:48.740239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:48.740239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:48.740239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:48.740239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:48.740239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:48.740239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:48.740239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:48.740239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:48.740239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:48.740239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:48.740239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:48.740792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:48.740792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:48.740792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.740792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:48.740968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.740968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:48.740968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:48.740968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:48.740968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:48.740968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:48.740968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:48.740968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:48.740968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:48.740968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:48.740968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:48.740968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:48.740968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:48.740968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:48.740968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:48.740968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:48.741534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:12:48.741534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:12:48.741534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:12:48.741534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:48.741534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:48.741534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:48.741534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:48.741534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:48.741534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:48.741534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:48.741534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:48.741534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:48.741534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:48.741534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:48.741534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:48.741534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:48.741534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:48.741534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:48.742202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:48.742202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:48.742202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:48.742202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:48.742202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:48.742202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.742202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:48.743177 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:48.743478 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:48.743595 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:48.743760 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:48.744362 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:48.744848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.745782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:48.745782 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:48.745782 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:48.745782 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:48.745782 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:48.745782 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:48.745782 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.745782 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:48.746649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.746649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:12:48.746649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:48.746649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:48.746649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:48.746649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:48.746649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:48.746649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:48.746649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:48.746649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:48.746649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:48.746649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:12:48.746649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:48.746649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:12:48.746649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:12:48.747215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:48.747215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:12:48.747215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:48.747215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:12:48.747215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:12:48.747215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:48.747215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:12:48.747215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.747215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:12:48.747894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.747894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:48.747894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:48.747894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:48.747894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:48.747894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:12:48.747894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:12:48.747894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:48.747894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:48.747894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:48.747894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:48.748299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:48.748299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:48.748299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:48.748299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:48.748299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:48.748299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:48.748299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:48.748299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:48.748299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:48.748299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:48.748299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:48.748299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:48.748299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:48.748299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:48.748863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:48.748863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:48.748863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.748863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:48.749291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.749291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:48.749291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:48.749291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:48.749291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:48.749291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:48.749291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:48.749291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:48.749291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:48.749291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:48.749291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:48.749291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:48.749291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:48.749291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:48.749291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:48.749291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:48.749291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:12:48.749926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:12:48.749926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:48.749926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:48.749926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:48.749926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:48.749926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:48.749926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:48.749926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:48.749926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:48.749926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:48.749926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:48.749926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:48.749926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:48.749926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:48.749926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:48.749926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:48.749926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:48.749926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:48.749926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:48.750673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:48.750673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:48.750673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.750673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:48.750851 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:48.751136 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:48.751293 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:48.751475 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:48.751921 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:48.752188 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:48.752317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.752932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:48.752932 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:48.752932 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:48.752932 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:48.752932 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:48.752932 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:48.752932 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.752932 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:48.753922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.753922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:48.753922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:48.753922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:48.753922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:48.753922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:12:48.753922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:48.753922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:48.753922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:48.753922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:48.753922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:48.753922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:48.754403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:48.754403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:48.754403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:48.754403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:48.754403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:48.754403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:48.754403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:48.754403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:48.754403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:48.754403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:48.754403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:48.754403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:48.754403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:48.754923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.754923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:48.755146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.755146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:48.755146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:48.755146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:48.755146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:48.755146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:48.755146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:48.755146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:48.755146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:48.755146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:48.755146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:48.755146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:48.755146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:48.755146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:48.755146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:48.755146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:48.755784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:12:48.755784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:48.755784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:48.755784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:48.755784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:48.755784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:48.755784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:48.755784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:48.755784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:48.755784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:48.755784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:48.755784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:48.755784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:48.755784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:48.755784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:48.755784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:48.755784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:48.755784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:48.755784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:48.756460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:48.756460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:48.756460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:52.937151 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:52.937959 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:52.937959 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:52.939482 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:52.993180 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:52.993649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:52.994715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:52.994715 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:52.994715 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:52.994715 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:52.994715 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:52.994715 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:52.994715 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:52.994715 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:52.996571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.057 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:52.996571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:52.996571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:52.996571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:52.996571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:52.996571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:12:52.996571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:52.996571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:52.996571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:52.996571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:52.996571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:52.996571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:52.997122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:52.997122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:52.997122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:52.997122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:52.997122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:52.997122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:52.997122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:52.997122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:52.997122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:52.997122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:52.997122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:52.997122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:52.997122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:52.997527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:52.997527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:52.998757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:52.998757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:52.998757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:52.998757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:52.998757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:52.998757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:52.998757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:52.998757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:52.998757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:52.998757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:52.998757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:52.998757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:52.998757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:52.998757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:52.998757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:52.998757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:52.999304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:12:52.999304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:52.999304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:52.999304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:52.999304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:52.999304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:52.999304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:52.999304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:52.999304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:52.999304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:52.999304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:52.999304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:52.999304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:52.999304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:52.999304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:52.999304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:52.999304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:52.999304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:52.999304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:52.999829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:52.999829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:52.999829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:53.714490 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:53.715162 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:53.715368 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:53.716402 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:53.716814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.717637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:53.717637 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:53.717637 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:53.717637 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:53.717637 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:53.717637 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:53.717637 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.717637 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:53.719230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.719230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:53.719230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:53.719230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:53.719230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:53.719230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:12:53.719230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:53.719230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:53.719230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:53.719230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:53.719230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:53.719230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:53.719973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:53.719973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:53.719973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:53.719973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:53.719973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:53.719973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:53.719973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:53.719973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:53.719973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:53.719973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:53.719973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:53.719973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:53.719973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:53.720641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.720641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:53.721027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.721027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:53.721027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:53.721027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:53.721027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:53.721027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:53.721027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:53.721027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:53.721027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:53.721027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:53.721027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:53.721027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:53.721027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:53.721027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:53.721027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:53.721027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:53.721896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:12:53.721896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:53.721896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:53.721896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:53.721896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:53.721896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:53.721896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:53.721896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:53.721896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:53.721896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:53.721896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:53.721896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:53.721896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:53.721896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:53.721896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:53.721896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:53.721896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:53.721896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:53.721896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:53.722872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:53.722872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.722872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:53.734243 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:53.734615 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:53.734895 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:53.735085 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:12:53.735592 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:53.735965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.736801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:53.736801 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:53.736801 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:53.736801 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:53.736801 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:53.736801 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:53.736801 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.736801 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:53.738084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.738084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:12:53.738084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:53.738084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:53.738084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:53.738084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:53.738084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:53.738084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:53.738084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:53.738084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:53.738084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:53.738084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:12:53.738084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:53.738084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:12:53.738084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:12:53.739278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:53.739278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:12:53.739278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:53.739278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:12:53.739278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:12:53.739278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:53.739278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:12:53.739278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.739278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:12:53.739918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.739918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:53.739918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:53.739918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:53.739918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:53.739918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:12:53.739918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:12:53.739918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:12:53.739918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:53.739918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:53.739918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:53.739918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:53.740940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:53.740940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:53.740940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:53.740940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:53.740940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:53.740940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:53.740940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:53.740940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:53.740940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:53.740940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:53.740940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:53.740940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:53.740940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:53.740940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:53.741885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:53.741885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:53.741885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.741885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:53.741885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.741885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:53.741885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:53.741885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:53.741885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:53.741885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:53.741885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:53.741885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:53.741885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:53.741885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:53.742408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:53.742408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:53.742408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:53.742408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:53.742408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:53.742408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:53.742408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:12:53.742408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:12:53.742408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:12:53.742408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:53.742408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:53.742408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:53.742408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:53.742408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:53.742408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:53.742408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:53.742408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:53.742408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:53.742408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:53.742915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:53.742915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:53.742915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:53.742915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:53.742915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:53.742915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:53.742915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:53.742915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:53.742915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:53.742915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:53.742915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.742915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:53.743603 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:53.744027 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:53.744164 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:53.744299 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:53.744772 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:53.745183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.746045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:53.746045 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:53.746045 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:53.746045 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:53.746045 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:53.746045 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:53.746045 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.746045 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:53.747255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.747255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:12:53.747255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:53.747255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:53.747255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:53.747255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:53.747255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:53.747255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:53.747255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:53.747255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:53.747255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:53.747255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:12:53.747255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:53.747255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:12:53.747255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:12:53.747899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:53.747899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:12:53.747899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:53.747899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:12:53.747899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:12:53.747899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:53.747899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:12:53.747899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.747899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:12:53.748877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.748877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:53.748877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:53.748877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:53.748877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:53.748877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:12:53.748877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:12:53.748877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:53.748877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:53.748877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:53.748877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:53.749208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:53.749208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:53.749208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:53.749208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:53.749208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:53.749208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:53.749208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:53.749208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:53.749208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:53.749208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:53.749208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:53.749208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:53.749208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:53.749208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:53.749628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:53.749628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:53.749628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.749628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:53.750797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.750797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:53.750797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:53.750797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:53.750797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:53.750797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:53.750797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:53.750797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:53.750797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:53.750797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:53.750797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:53.750797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:53.750797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:53.750797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:53.750797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:53.750797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:53.750797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:12:53.751284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:12:53.751284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:53.751284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:53.751284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:53.751284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:53.751284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:53.751284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:53.751284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:53.751284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:53.751284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:53.751284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:53.751284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:53.751284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:53.751284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:53.751284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:53.751284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:53.751284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:53.751284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:53.751284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:53.751865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:53.751865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:53.751865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.751865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:53.751865 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:53.751865 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:53.751865 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:53.752084 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:53.752516 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:53.752888 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:53.753100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.753904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:53.753904 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:53.753904 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:53.753904 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:53.753904 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:53.753904 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:53.753904 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.753904 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:53.754874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.754874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:53.754874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:53.754874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:53.754874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:53.754874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:12:53.754874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:53.754874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:53.754874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:53.754874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:53.754874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:53.754874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:53.755376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:53.755376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:53.755376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:53.755376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:53.755376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:53.755376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:53.755376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:53.755376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:53.755376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:53.755376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:53.755376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:53.755376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:53.755376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:53.756001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.756001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:53.756001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.756001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:53.756001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:53.756001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:53.756001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:53.756001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:53.756001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:53.756001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:53.756001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:53.756001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:53.756001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:53.756001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:53.756001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:53.756598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:53.756598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:53.756598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:53.756598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:12:53.756598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:53.756598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:53.756598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:53.756598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:53.756598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:53.756598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:53.756598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:53.756598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:53.756598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:53.756598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:53.756598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:53.756598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:53.756598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:53.756598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:53.756598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:53.757250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:53.757250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:53.757250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:53.757250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:53.757250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:53.757250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:57.937810 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:57.938523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:57.938523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:57.939834 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:57.974291 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:57.974716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:57.975751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:57.975751 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:57.975751 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:57.975751 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:57.975751 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:57.975751 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:57.975751 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:57.975751 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:57.976672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:57.976672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:57.976672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:57.976672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:57.976672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:57.976672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:12:57.976672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:57.976672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:57.976672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:57.976672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:57.976672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:57.976672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:57.977296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:57.977296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:57.977296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:57.977296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:57.977296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:57.977296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:57.977296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:57.977296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:57.977296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:57.977296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:57.977296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:57.977296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:57.977296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:57.977918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:57.977918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:57.978122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:57.978122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:57.978122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:57.978122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:57.978122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:57.978122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:57.978122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:57.978122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:57.978122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:57.978122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:57.978122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:57.978122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:57.978122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:57.978122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:57.978122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:57.978122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:57.978907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:12:57.978907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:57.978907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:57.978907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:57.978907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:57.978907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:57.978907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:57.978907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:57.978907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:57.978907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:57.978907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:57.978907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:57.978907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:57.978907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:57.978907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:57.978907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:57.978907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:57.978907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:57.978907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:57.979774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:57.979774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:57.979774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:58.714985 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:58.715626 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:58.715877 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:58.716895 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:58.717214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.718385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:58.718385 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:58.718385 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:58.718385 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:58.718385 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:58.718385 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:58.718385 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.718385 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:58.720376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.720376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:58.720376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:58.720376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:58.720376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:58.720376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:12:58.720376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:58.720376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:58.720376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:58.720376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:58.720376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:58.720376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:58.721225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:58.721225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:58.721225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:58.721225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:58.721225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:58.721225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:58.721225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:58.721225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:58.721225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:58.721225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:58.721225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:58.721225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:58.721225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:58.722067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.722067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:58.722577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.722577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:58.722577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:58.722577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:58.722577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:58.722577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:58.722577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:58.722577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:58.722577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:58.722577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:58.722577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:58.722577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:58.722577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:58.722577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:58.722577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:58.722577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:58.723999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:12:58.723999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:58.723999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:58.723999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:58.723999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:58.723999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:58.723999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:58.723999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:58.723999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:58.723999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:58.723999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:58.723999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:58.723999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:58.723999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:58.723999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:58.723999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:58.723999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:58.723999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:58.723999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:58.724704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:58.724704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.724704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:58.734776 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:58.735211 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:58.735468 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:58.735670 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:12:58.736711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:58.737078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.738194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:58.738194 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:58.738194 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:58.738194 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:58.738194 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:58.738194 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:58.738194 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.738194 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:58.739508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.739508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:12:58.739508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:58.739508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:58.739508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:58.739508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:58.739508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:58.739508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:58.739508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:58.739508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:58.739508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:58.739508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:12:58.739508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:58.739508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:12:58.739508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:12:58.740348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:58.740348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:12:58.740348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:58.740348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:12:58.740348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:12:58.740348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:58.740348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:12:58.740348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.740348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:12:58.741000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.741000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:58.741000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:58.741000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:58.741000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:58.741000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:12:58.741000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:12:58.741000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:12:58.741000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:58.741000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:58.741000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:58.741000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:58.741589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:58.741589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:58.741589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:58.741589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:58.741589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:58.741589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:58.741589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:58.741589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:58.741589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:58.741589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:58.741589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:58.741589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:58.741589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:58.741589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:58.742267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:58.742267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:58.742267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.742267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:58.742801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.742801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:58.742801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:58.742801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:58.742801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:58.742801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:58.742801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:58.742801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:58.742801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:58.742801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:58.742801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:58.742801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:58.742801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:58.742801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:58.742801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:58.742801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:58.743636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:12:58.743636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:12:58.743636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:12:58.743636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:58.743636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:58.743636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:58.743636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:58.743636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:58.743636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:58.743636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:58.743636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:58.743636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:58.743636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:58.743636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:58.743636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:58.743636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:58.743636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:58.743636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:58.744733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:58.744733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:58.744733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:58.744733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:58.744733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:58.744733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.744733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:58.744733 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:58.744733 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:58.744733 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:58.745179 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:58.745539 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:58.745834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.746465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:58.746465 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:58.746465 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:58.746465 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:58.746465 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:58.746465 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:58.746465 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.746465 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:58.747777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.747777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:12:58.747777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:58.747777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:58.747777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:58.747777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:58.747777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:58.747777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:58.747777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:58.747777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:12:58.747777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:58.747777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:12:58.747777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:58.747777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:12:58.747777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:12:58.748413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:58.748413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:12:58.748413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:58.748413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:12:58.748413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:12:58.748413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:58.748413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:12:58.748413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.748413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:12:58.748900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.748900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:58.748900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:58.748900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:58.748900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:58.748900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:12:58.748900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:12:58.748900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:58.748900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:58.748900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:58.748900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:58.749221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:58.749221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:58.749221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:58.749221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:58.749221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:58.749221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:58.749221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:58.749221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:58.749221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:58.749221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:58.749221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:58.749221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:58.749221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:58.749221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:58.749705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:58.749705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:58.749705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.749705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:58.749953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.749953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:58.749953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:58.749953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:58.749953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:58.749953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:58.749953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:58.749953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:58.749953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:58.749953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:58.749953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:58.749953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:58.749953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:58.749953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:58.749953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:58.749953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:58.749953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:12:58.750476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:12:58.750476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:12:58.750476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:12:58.750476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:58.750476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:58.750476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:58.750476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:58.750476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:58.750476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:58.750476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:58.750476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:58.750476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:58.750476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:58.750476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:58.750476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:58.750476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:58.750476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:58.750476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:58.750476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:58.751337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:58.751337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:58.751337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.751337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:12:58.751617 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:12:58.751897 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:12:58.752012 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:12:58.752185 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:12:58.752874 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:58.753226 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:12:58.753422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.754839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:12:58.754839 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:58.754839 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:12:58.754839 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:12:58.754839 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:58.754839 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:12:58.754839 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.754839 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:12:58.755742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.755742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:12:58.755742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:58.755742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:12:58.755742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:58.755742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:12:58.755742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:58.755742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:58.755742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:58.755742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:58.755742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:58.755742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:58.756557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:12:58.756557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:58.756557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:12:58.756557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:58.756557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:12:58.756557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:12:58.756557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:58.756557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:12:58.756557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:58.756557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:12:58.756557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:12:58.756557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:58.756557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:12:58.757240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.757240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:12:58.757240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.757240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:12:58.757240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:12:58.757240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:12:58.757240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:12:58.757240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:12:58.757240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:58.757240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:12:58.757240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:12:58.757240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:58.757240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:12:58.757240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:12:58.757240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:12:58.758002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:12:58.758002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:12:58.758002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:12:58.758002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:12:58.758002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:12:58.758002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:58.758002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:58.758002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:12:58.758002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:12:58.758002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:12:58.758002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:12:58.758002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:12:58.758002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:12:58.758002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:12:58.758002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:12:58.758002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:12:58.758002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:12:58.758002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:12:58.758002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:12:58.759083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:12:58.759083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:12:58.759083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:12:58.759083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:12:58.759083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:12:58.759083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:02.938758 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:02.939049 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:02.939732 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:02.940427 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:02.961892 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=255 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:13:02.970219 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:02.970219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:02.970539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:02.970539 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:02.970539 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:02.970539 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:02.970539 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:02.970539 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:02.970539 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:02.970539 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:02.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:02.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:02.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:02.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:02.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:02.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:13:02.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:02.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:02.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:02.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:02.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:02.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:02.971569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:02.971569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:02.971569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:02.971569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:02.971569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:02.971569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:02.971569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:02.971569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:02.971569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:02.971569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:02.971569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:02.971569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:02.971569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:02.971847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:02.971847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:02.972936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:02.972936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:02.972936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:02.972936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:02.972936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:02.972936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:02.972936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:02.972936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:02.972936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:02.972936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:02.972936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:02.972936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:02.972936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:02.972936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:02.972936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:02.972936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:02.973985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:13:02.973985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:02.973985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:02.973985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:02.973985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:02.973985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:02.973985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:02.973985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:02.973985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:02.973985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:02.973985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:02.973985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:02.973985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:02.973985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:02.973985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:02.973985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:02.973985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:02.973985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:02.973985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:02.975095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:02.975095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:02.975095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:03.717725 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:03.717725 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:03.717725 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:03.717725 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:03.717725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.718346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:03.718346 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:03.718346 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:03.718346 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:03.718346 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:03.718346 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:03.718346 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.718346 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:03.720050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.720050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:03.720050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:03.720050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:03.720050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:03.720050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:13:03.720050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:03.720050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:03.720050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:03.720050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:03.720050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:03.720050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:03.720396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:03.720396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:03.720396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:03.720396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:03.720396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:03.720396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:03.720396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:03.720396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:03.720396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:03.720396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:03.720396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:03.720396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:03.720396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:03.720745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.720745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:03.721929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.721929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:03.721929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:03.721929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:03.721929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:03.721929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:03.721929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:03.721929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:03.721929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:03.721929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:03.721929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:03.721929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:03.721929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:03.721929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:03.721929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:03.721929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:03.722368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:13:03.722368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:03.722368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:03.722368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:03.722368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:03.722368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:03.722368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:03.722368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:03.722368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:03.722368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:03.722368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:03.722368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:03.722368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:03.722368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:03.722368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:03.722368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:03.722368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:03.722368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:03.722368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:03.722949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:03.722949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.722949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:03.735313 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:03.735590 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:03.735777 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:03.735962 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:13:03.736430 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:03.736661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.737557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:03.737557 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:03.737557 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:03.737557 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:03.737557 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:03.737557 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:03.737557 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.737557 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:03.738762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.738762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:13:03.738762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:03.738762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:03.738762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:03.738762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:03.738762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:03.738762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:03.738762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:03.738762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:03.738762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:03.738762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:13:03.738762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:03.738762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:13:03.738762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:13:03.739546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:03.739546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:13:03.739546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:03.739546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:13:03.739546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:13:03.739546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:03.739546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:13:03.739546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.739546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:13:03.740114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.740114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:03.740114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:03.740114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:03.740114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:03.740114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:13:03.740114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:13:03.740114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:13:03.740114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:03.740114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:03.740114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:03.740114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:03.740644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:03.740644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:03.740644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:03.740644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:03.740644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:03.740644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:03.740644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:03.740644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:03.740644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:03.740644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:03.740644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:03.740644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:03.740644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:03.740644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:03.741309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:03.741309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:03.741309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.741309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:03.741875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.741875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:03.741875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:03.741875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:03.741875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:03.741875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:03.741875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:03.741875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:03.741875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:03.741875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:03.741875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:03.741875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:03.741875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:03.741875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:03.741875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:03.741875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:03.743193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:13:03.743193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:13:03.743193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:13:03.743193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:03.743193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:03.743193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:03.743193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:03.743193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:03.743193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:03.743193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:03.743193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:03.743193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:03.743193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:03.743193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:03.743193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:03.743193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:03.743193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:03.743193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:03.744210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:03.744210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:03.744210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:03.744210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:03.744210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:03.744210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.744210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:03.744704 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:03.744704 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:03.745053 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:03.745272 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:03.745625 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:03.745865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.746422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:03.746422 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:03.746422 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:03.746422 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:03.746422 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:03.746422 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:03.746422 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.746422 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:03.747473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.747473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:13:03.747473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:03.747473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:03.747473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:03.747473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:03.747473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:03.747473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:03.747473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:03.747473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:03.747473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:03.747473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:13:03.747473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:03.747473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:13:03.747473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:13:03.748149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:03.748149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:13:03.748149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:03.748149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:13:03.748149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:13:03.748149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:03.748149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:13:03.748149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.748149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:13:03.748508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.748508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:03.748508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:03.748508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:03.748508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:03.748508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:13:03.748508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:13:03.748508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:03.748508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:03.748508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:03.748508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:03.749096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:03.749096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:03.749096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:03.749096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:03.749096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:03.749096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:03.749096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:03.749096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:03.749096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:03.749096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:03.749096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:03.749096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:03.749096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:03.749096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:03.749653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:03.749653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:03.749653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.749653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:03.749653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.749653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:03.749653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:03.749653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:03.749653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:03.749653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:03.749653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:03.749653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:03.749653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:03.749653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:03.750240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:03.750240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:03.750240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:03.750240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:03.750240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:03.750240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:03.750240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:13:03.750240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:13:03.750240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:03.750240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:03.750240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:03.750240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:03.750240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:03.750240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:03.750240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:03.750240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:03.750240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:03.750240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:03.750240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:03.751226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:03.751226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:03.751226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:03.751226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:03.751226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:03.751226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:03.751226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:03.751226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:03.751226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:03.751226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.751226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:03.752100 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:03.752392 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:03.752494 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:03.752653 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:03.753195 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:03.753475 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:03.753631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.754332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:03.754332 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:03.754332 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:03.754332 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:03.754332 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:03.754332 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:03.754332 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.754332 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:03.755423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.755423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:03.755423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:03.755423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:03.755423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:03.755423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:13:03.755423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:03.755423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:03.755423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:03.755423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:03.755423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:03.755423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:03.755893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:03.755893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:03.755893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:03.755893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:03.755893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:03.755893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:03.755893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:03.755893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:03.755893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:03.755893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:03.755893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:03.755893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:03.755893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:03.756229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.756229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:03.756511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.756511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:03.756511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:03.756511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:03.756511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:03.756511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:03.756511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:03.756511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:03.756511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:03.756511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:03.756511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:03.756511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:03.756511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:03.756511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:03.756511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:03.756511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:03.756996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:13:03.756996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:03.756996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:03.756996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:03.756996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:03.756996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:03.756996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:03.756996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:03.756996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:03.756996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:03.756996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:03.756996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:03.756996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:03.756996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:03.756996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:03.756996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:03.756996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:03.756996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:03.756996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:03.757493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:03.757493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:03.757493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:07.939230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:07.940061 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:07.940061 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:07.941292 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:07.969650 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:07.970080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:07.970906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:07.970906 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:07.970906 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:07.970906 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:07.970906 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:07.970906 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:07.970906 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:07.970906 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:07.971970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:07.971970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:07.971970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:07.971970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:07.971970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:07.971970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:13:07.971970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:07.971970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:07.971970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:07.971970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:07.971970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:07.971970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:07.972505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:07.972505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:07.972505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:07.972505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:07.972505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:07.972505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:07.972505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:07.972505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:07.972505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:07.972505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:07.972505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:07.972505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:07.972505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:07.973062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:07.973062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:07.973062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:07.973062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:07.973062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:07.973062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:07.973062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:07.973062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:07.973062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:07.973062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:07.973062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:07.973062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:07.973062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:07.973062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:07.973062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:07.973834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:07.973834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:07.973834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:07.973834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:13:07.973834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:07.973834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:07.973834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:07.973834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:07.973834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:07.973834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:07.973834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:07.973834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:07.973834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:07.973834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:07.973834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:07.973834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:07.973834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:07.973834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:07.973834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:07.974570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:07.974570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:07.974570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:07.974570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:07.974570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:07.974570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:08.716479 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:08.716772 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:08.716911 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:08.717777 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:08.718118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.719007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:08.719007 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:08.719007 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:08.719007 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:08.719007 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:08.719007 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:08.719007 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.719007 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:08.720550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.720550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:08.720550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:08.720550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:08.720550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:08.720550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:13:08.720550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:08.720550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:08.720550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:08.720550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:08.720550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:08.720550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:08.721083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:08.721083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:08.721083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:08.721083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:08.721083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:08.721083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:08.721083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:08.721083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:08.721083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:08.721083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:08.721083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:08.721083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:08.721083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:08.721608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.721608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:08.722447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.722447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:08.722447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:08.722447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:08.722447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:08.722447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:08.722447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:08.722447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:08.722447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:08.722447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:08.722447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:08.722447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:08.722447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:08.722447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:08.722447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:08.722447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:08.723024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:13:08.723024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:08.723024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:08.723024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:08.723024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:08.723024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:08.723024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:08.723024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:08.723024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:08.723024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:08.723024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:08.723024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:08.723024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:08.723024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:08.723024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:08.723024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:08.723024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:08.723024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:08.723024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:08.723536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:08.723536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.723536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:08.735813 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:08.736195 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:08.736389 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:08.736612 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:13:08.737143 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:08.737438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.738417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:08.738417 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:08.738417 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:08.738417 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:08.738417 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:08.738417 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:08.738417 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.738417 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:08.739721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.739721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:13:08.739721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:08.739721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:08.739721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:08.739721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:08.739721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:08.739721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:08.739721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:08.739721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:08.739721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:08.739721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:13:08.739721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:08.739721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:13:08.739721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:13:08.740337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:08.740337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:13:08.740337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:08.740337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:13:08.740337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:13:08.740337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:08.740337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:13:08.740337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.740337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:13:08.741204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.741204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:08.741204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:08.741204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:08.741204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:08.741204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:13:08.741204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:13:08.741204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:13:08.741204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:08.741204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:08.741204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:08.741204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:08.741744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:08.741744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:08.741744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:08.741744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:08.741744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:08.741744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:08.741744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:08.741744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:08.741744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:08.741744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:08.741744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:08.741744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:08.741744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:08.741744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:08.742320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:08.742320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:08.742320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.742320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:08.743034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.743034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:08.743034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:08.743034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:08.743034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:08.743034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:08.743034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:08.743034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:08.743034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:08.743034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:08.743034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:08.743034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:08.743034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:08.743034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:08.743034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:08.743034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:08.743760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:13:08.743760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:13:08.743760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:13:08.743760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:08.743760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:08.743760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:08.743760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:08.743760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:08.743760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:08.743760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:08.743760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:08.743760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:08.743760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:08.743760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:08.743760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:08.743760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:08.743760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:08.743760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:08.744604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:08.744604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:08.744604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:08.744604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:08.744604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:08.744604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.744604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:08.745028 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:08.745218 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:08.745363 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:08.745555 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:08.745979 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:08.746222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.746820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:08.746820 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:08.746820 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:08.746820 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:08.746820 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:08.746820 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:08.746820 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.746820 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:08.747657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.747657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:13:08.747657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:08.747657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:08.747657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:08.747657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:08.747657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:08.747657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:08.747657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:08.747657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:08.747657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:08.747657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:13:08.747657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:08.747657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:13:08.747657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:13:08.748352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:08.748352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:13:08.748352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:08.748352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:13:08.748352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:13:08.748352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:08.748352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:13:08.748352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.748352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:13:08.749003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.749003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:08.749003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:08.749003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:08.749003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:08.749003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:13:08.749003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:13:08.749003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:08.749003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:08.749003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:08.749003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:08.749445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:08.749445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:08.749445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:08.749445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:08.749445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:08.749445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:08.749445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:08.749445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:08.749445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:08.749445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:08.749445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:08.749445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:08.749445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:08.749445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:08.750008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:08.750008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:08.750008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.750008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:08.750532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.750532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:08.750532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:08.750532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:08.750532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:08.750532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:08.750532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:08.750532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:08.750532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:08.750532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:08.750532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:08.750532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:08.750532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:08.750532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:08.750532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:08.750532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:08.750532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:13:08.751002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:13:08.751002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:08.751002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:08.751002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:08.751002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:08.751002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:08.751002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:08.751002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:08.751002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:08.751002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:08.751002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:08.751002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:08.751002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:08.751002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:08.751002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:08.751002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:08.751002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:08.751002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:08.751002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:08.751478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:08.751478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:08.751478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.751478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:08.752406 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:08.752633 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:08.752806 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:08.753000 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:08.753267 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:08.753509 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:08.753639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.754214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:08.754214 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:08.754214 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:08.754214 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:08.754214 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:08.754214 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:08.754214 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.754214 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:08.755141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.755141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:08.755141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:08.755141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:08.755141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:08.755141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:13:08.755141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:08.755141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:08.755141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:08.755141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:08.755141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:08.755141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:08.755765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:08.755765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:08.755765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:08.755765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:08.755765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:08.755765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:08.755765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:08.755765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:08.755765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:08.755765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:08.755765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:08.755765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:08.755765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:08.756296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.756296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:08.756525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.756525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:08.756525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:08.756525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:08.756525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:08.756525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:08.756525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:08.756525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:08.756525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:08.756525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:08.756525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:08.756525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:08.756525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:08.756525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:08.756525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:08.756525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:08.757005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:13:08.757005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:08.757005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:08.757005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:08.757005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:08.757005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:08.757005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:08.757005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:08.757005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:08.757005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:08.757005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:08.757005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:08.757005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:08.757005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:08.757005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:08.757005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:08.757005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:08.757005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:08.757005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:08.757461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:08.757461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:08.757461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:12.940011 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:12.940761 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:12.940854 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:12.942320 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:12.972726 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:12.973170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:12.974017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:12.974017 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:12.974017 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:12.974017 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:12.974017 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:12.974017 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:12.974017 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:12.974017 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:12.974858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:12.974858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:12.974858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:12.974858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:12.974858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:12.974858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:13:12.974858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:12.974858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:12.974858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:12.974858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:12.974858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:12.974858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:12.975398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:12.975398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:12.975398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:12.975398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:12.975398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:12.975398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:12.975398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:12.975398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:12.975398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:12.975398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:12.975398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:12.975398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:12.975398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:12.976219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:12.976219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:12.976219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:12.976219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:12.976219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:12.976219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:12.976219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:12.976219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:12.976219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:12.976219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:12.976219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:12.976219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:12.976219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:12.976219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:12.976219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:12.976874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:12.976874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:12.976874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:12.976874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:13:12.976874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:12.976874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:12.976874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:12.976874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:12.976874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:12.976874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:12.976874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:12.976874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:12.976874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:12.976874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:12.976874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:12.976874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:12.976874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:12.976874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:12.976874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:12.977809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:12.977809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:12.977809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:12.977809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:12.977809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:12.977809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:13.717092 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:13.717824 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:13.718029 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:13.719107 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:13.719444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.720423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:13.720423 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:13.720423 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:13.720423 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:13.720423 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:13.720423 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:13.720423 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.720423 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:13.722043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.722043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:13.722043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:13.722043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:13.722043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:13.722043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:13:13.722043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:13.722043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:13.722043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:13.722043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:13.722043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:13.722043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:13.722651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:13.722651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:13.722651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:13.722651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:13.722651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:13.722651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:13.722651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:13.722651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:13.722651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:13.722651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:13.722651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:13.722651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:13.722651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:13.723271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.723271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:13.723693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.723693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:13.723693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:13.723693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:13.723693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:13.723693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:13.723693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:13.723693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:13.723693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:13.723693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:13.723693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:13.723693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:13.723693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:13.723693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:13.723693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:13.723693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:13.724208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:13:13.724208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:13.724208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:13.724208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:13.724208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:13.724208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:13.724208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:13.724208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:13.724208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:13.724208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:13.724208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:13.724208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:13.724208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:13.724208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:13.724208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:13.724208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:13.724208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:13.724208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:13.724208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:13.725015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:13.725015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.725015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:13.736261 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:13.736642 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:13.736858 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:13.737047 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:13:13.737588 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:13.737976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.738712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:13.738712 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:13.738712 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:13.738712 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:13.738712 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:13.738712 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:13.738712 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.738712 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:13.739888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.739888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:13:13.739888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:13.739888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:13.739888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:13.739888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:13.739888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:13.739888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:13.739888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:13.739888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:13.739888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:13.739888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:13:13.739888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:13.739888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:13:13.739888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:13:13.740566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:13.740566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:13:13.740566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:13.740566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:13:13.740566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:13:13.740566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:13.740566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:13:13.740566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.740566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:13:13.741241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.741241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:13.741241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:13.741241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:13.741241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:13.741241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:13:13.741241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:13:13.741241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:13:13.741241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:13.741241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:13.741241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:13.741241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:13.742054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:13.742054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:13.742054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:13.742054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:13.742054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:13.742054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:13.742054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:13.742054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:13.742054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:13.742054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:13.742054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:13.742054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:13.742054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:13.742054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:13.742586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:13.742586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:13.742586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.742586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:13.742789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.742789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:13.742789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:13.742789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:13.742789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:13.742789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:13.742789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:13.742789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:13.742789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:13.742789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:13.742789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:13.742789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:13.742789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:13.742789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:13.742789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:13.742789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:13.743209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:13:13.743209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:13:13.743209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:13:13.743209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:13.743209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:13.743209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:13.743209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:13.743209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:13.743209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:13.743209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:13.743209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:13.743209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:13.743209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:13.743209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:13.743209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:13.743209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:13.743209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:13.743209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:13.743694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:13.743694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:13.743694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:13.743694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:13.743694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:13.743694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.743694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:13.745169 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:13.745419 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:13.745557 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:13.745691 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:13.746151 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:13.746494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.747185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:13.747185 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:13.747185 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:13.747185 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:13.747185 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:13.747185 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:13.747185 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.747185 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:13.747927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.747927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:13:13.747927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:13.747927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:13.747927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:13.747927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:13.747927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:13.747927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:13.747927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:13.747927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:13.747927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:13.747927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:13:13.747927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:13.747927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:13:13.747927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:13:13.748317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:13.748317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:13:13.748317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:13.748317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:13:13.748317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:13:13.748317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:13.748317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:13:13.748317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.748317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:13:13.748800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.748800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:13.748800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:13.748800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:13.748800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:13.748800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:13:13.748800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:13:13.748800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:13.748800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:13.748800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:13.748800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:13.749100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:13.749100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:13.749100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:13.749100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:13.749100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:13.749100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:13.749100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:13.749100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:13.749100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:13.749100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:13.749100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:13.749100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:13.749100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:13.749100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:13.749471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:13.749471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:13.749471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.749471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:13.749915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.749915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:13.749915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:13.749915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:13.749915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:13.749915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:13.749915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:13.749915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:13.749915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:13.749915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:13.749915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:13.749915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:13.749915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:13.749915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:13.749915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:13.749915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:13.749915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:13:13.750402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:13:13.750402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:13.750402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:13.750402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:13.750402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:13.750402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:13.750402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:13.750402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:13.750402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:13.750402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:13.750402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:13.750402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:13.750402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:13.750402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:13.750402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:13.750402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:13.750402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:13.750402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:13.750402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:13.750941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:13.750941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:13.750941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.750941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:13.752820 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:13.753103 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:13.753210 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:13.753360 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:13.753819 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:13.754163 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:13.754337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.755015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:13.755015 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:13.755015 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:13.755015 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:13.755015 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:13.755015 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:13.755015 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.755015 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:13.755831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.755831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:13.755831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:13.755831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:13.755831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:13.755831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:13:13.755831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:13.755831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:13.755831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:13.755831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:13.755831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:13.755831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:13.756190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:13.756190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:13.756190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:13.756190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:13.756190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:13.756190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:13.756190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:13.756190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:13.756190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:13.756190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:13.756190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:13.756190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:13.756190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:13.756569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.756569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:13.756812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.756812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:13.756812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:13.756812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:13.756812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:13.756812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:13.756812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:13.756812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:13.756812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:13.756812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:13.756812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:13.756812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:13.756812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:13.756812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:13.756812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:13.756812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:13.757278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:13:13.757278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:13.757278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:13.757278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:13.757278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:13.757278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:13.757278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:13.757278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:13.757278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:13.757278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:13.757278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:13.757278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:13.757278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:13.757278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:13.757278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:13.757278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:13.757278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:13.757278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:13.757278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:13.757883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:13.757883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:13.757883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:17.941646 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:17.941646 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:17.941646 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:17.943725 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:17.969610 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:17.969995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:17.970750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:17.970750 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:17.970750 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:17.970750 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:17.970750 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:17.970750 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:17.970750 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:17.970750 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:17.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:17.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:17.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:17.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:17.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:17.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:13:17.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:17.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:17.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:17.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:17.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:17.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:17.972150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:17.972150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:17.972150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:17.972150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:17.972150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:17.972150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:17.972150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:17.972150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:17.972150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:17.972150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:17.972150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:17.972150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:17.972150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:17.972661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:17.972661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:17.972661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:17.972661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:17.972661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:17.972661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:17.972661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:17.972661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:17.972661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:17.972661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:17.972661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:17.972661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:17.972661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:17.972661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:17.972661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:17.973530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:17.973530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:17.973530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:17.973530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:13:17.973530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:17.973530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:17.973530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:17.973530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:17.973530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:17.973530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:17.973530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:17.973530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:17.973530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:17.973530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:17.973530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:17.973530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:17.973530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:17.973530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:17.973530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:17.974539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:17.974539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:17.974539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:17.974539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:17.974539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:17.974539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:18.717650 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:18.717966 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:18.718060 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:18.718936 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:18.719198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.719855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:18.719855 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:18.719855 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:18.719855 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:18.719855 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:18.719855 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:18.719855 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.719855 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:18.720824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.720824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:18.720824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:18.720824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:18.720824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:18.720824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:13:18.720824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:18.720824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:18.720824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:18.720824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:18.720824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:18.720824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:18.721582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:18.721582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:18.721582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:18.721582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:18.721582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:18.721582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:18.721582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:18.721582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:18.721582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:18.721582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:18.721582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:18.721582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:18.721582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:18.722416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.722416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:18.722416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.722416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:18.722416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:18.722416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:18.722416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:18.722416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:18.722416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:18.722416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:18.722416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:18.722416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:18.722416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:18.722416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:18.722416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:18.723420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:18.723420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:18.723420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:18.723420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:13:18.723420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:18.723420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:18.723420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:18.723420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:18.723420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:18.723420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:18.723420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:18.723420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:18.723420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:18.723420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:18.723420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:18.723420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:18.723420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:18.723420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:18.723420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:18.725497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:18.725497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:18.725497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:18.725497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:18.725497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.725497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:18.736629 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:18.737019 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:18.737208 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:18.737392 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:13:18.738043 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:18.738367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.739262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:18.739262 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:18.739262 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:18.739262 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:18.739262 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:18.739262 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:18.739262 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.739262 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:18.741135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.741135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:13:18.741135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:18.741135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:18.741135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:18.741135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:18.741135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:18.741135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:18.741135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:18.741135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:18.741135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:18.741135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:13:18.741135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:18.741135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:13:18.741135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:13:18.742218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:18.742218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:13:18.742218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:18.742218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:13:18.742218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:13:18.742218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:18.742218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:13:18.742218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.742218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:13:18.742218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.742218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:18.742218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:18.742218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:18.742218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:18.743199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:13:18.743199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:13:18.743199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:13:18.743199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:18.743199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:18.743199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:18.743199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:18.743199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:18.743199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:18.743199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:18.743199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:18.743199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:18.743199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:18.743806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:18.743806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:18.743806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:18.743806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:18.743806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:18.743806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:18.743806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:18.743806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:18.743806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:18.743806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:18.743806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.743806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:18.744290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.744290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:18.744290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:18.744290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:18.744290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:18.744290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:18.744290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:18.744290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:18.744290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:18.744290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:18.744290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:18.744290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:18.744290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:18.744290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:18.744290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:18.744290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:18.744931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:13:18.744931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:13:18.744931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:13:18.744931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:18.744931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:18.744931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:18.744931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:18.744931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:18.744931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:18.744931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:18.744931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:18.744931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:18.744931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:18.744931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:18.744931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:18.744931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:18.744931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:18.744931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:18.745648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:18.745648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:18.745648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:18.745648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:18.745648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:18.745648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.745648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:18.745648 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:18.746004 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:18.746004 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:18.746151 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:18.746632 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:18.747038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.747726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:18.747726 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:18.747726 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:18.747726 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:18.747726 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:18.747726 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:18.747726 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.747726 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:18.748526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.748526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:13:18.748526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:18.748526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:18.748526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:18.748526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:18.748526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:18.748526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:18.748526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:18.748526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:18.748526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:18.748526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:13:18.748526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:18.748526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:13:18.748526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:13:18.749113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:18.749113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:13:18.749113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:18.749113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:13:18.749113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:13:18.749113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:18.749113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:13:18.749113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.749113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:13:18.749565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.749565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:18.749565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:18.749565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:18.749565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:18.749565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:13:18.749565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:13:18.749565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:18.749565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:18.749565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:18.749565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:18.750049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:18.750049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:18.750049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:18.750049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:18.750049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:18.750049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:18.750049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:18.750049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:18.750049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:18.750049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:18.750049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:18.750049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:18.750049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:18.750049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:18.750655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:18.750655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:18.750655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.750655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:18.750655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.750655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:18.750655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:18.750655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:18.750655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:18.750655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:18.750655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:18.750655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:18.750655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:18.750655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:18.751241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:18.751241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:18.751241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:18.751241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:18.751241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:18.751241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:18.751241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:13:18.751241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:13:18.751241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:18.751241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:18.751241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:18.751241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:18.751241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:18.751241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:18.751241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:18.751241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:18.751241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:18.751241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:18.751241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:18.751999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:18.751999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:18.751999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:18.751999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:18.751999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:18.751999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:18.751999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:18.751999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:18.751999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:18.751999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.751999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:18.753166 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:18.753396 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:18.753514 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:18.753708 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:18.754189 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:18.754539 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:18.754843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.755530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:18.755530 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:18.755530 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:18.755530 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:18.755530 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:18.755530 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:18.755530 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.755530 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:18.756481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.756481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:18.756481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:18.756481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:18.756481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:18.756481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:13:18.756481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:18.756481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:18.756481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:18.756481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:18.756481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:18.756481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:18.756997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:18.756997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:18.756997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:18.756997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:18.756997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:18.756997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:18.756997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:18.756997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:18.756997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:18.756997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:18.756997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:18.756997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:18.756997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:18.757895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.757895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:18.757895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.757895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:18.757895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:18.757895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:18.757895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:18.757895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:18.757895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:18.757895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:18.757895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:18.757895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:18.757895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:18.757895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:18.757895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:18.758790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:18.758790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:18.758790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:18.758790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:13:18.758790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:18.758790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:18.758790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:18.758790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:18.758790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:18.758790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:18.758790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:18.758790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:18.758790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:18.758790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:18.758790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:18.758790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:18.758790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:18.758790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:18.758790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:18.759540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:18.759540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:18.759540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:18.759540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:18.759540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:18.759540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:22.942053 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:22.942655 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:22.942900 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:22.944152 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:22.977055 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:22.977547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:22.978333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:22.978333 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:22.978333 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:22.978333 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:22.978333 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:22.978333 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:22.978333 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:22.978333 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:22.979072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:22.979072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:22.979072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:22.979072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:22.979072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:22.979072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:13:22.979072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:22.979072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:22.979072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:22.979072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:22.979072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:22.979072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:22.979371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:22.979371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:22.979371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:22.979371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:22.979371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:22.979371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:22.979371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:22.979371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:22.979371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:22.979371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:22.979371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:22.979371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:22.979371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:22.979664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:22.979664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:22.979863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:22.979863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:22.979863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:22.979863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:22.979863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:22.979863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:22.979863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:22.979863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:22.979863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:22.979863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:22.979863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:22.979863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:22.979863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:22.979863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:22.979863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:22.979863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:22.980504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:13:22.980504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:22.980504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:22.980504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:22.980504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:22.980504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:22.980504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:22.980504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:22.980504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:22.980504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:22.980504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:22.980504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:22.980504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:22.980504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:22.980504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:22.980504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:22.980504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:22.980504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:22.980504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:22.981300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:22.981300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:22.981300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:23.718259 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:23.718771 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:23.718905 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:23.719670 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:23.722944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.723528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:23.723528 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:23.723528 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:23.723528 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:23.723528 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:23.723528 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:23.723528 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.723528 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:23.724558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.724558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:23.724558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:23.724558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:23.724558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:23.724558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:13:23.724558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:23.724558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:23.724558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:23.724558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:23.724558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:23.724558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:23.725167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:23.725167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:23.725167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:23.725167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:23.725167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:23.725167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:23.725167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:23.725167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:23.725167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:23.725167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:23.725167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:23.725167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:23.725167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:23.725830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.725830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:23.725830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.725830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:23.725830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:23.725830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:23.725830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:23.725830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:23.725830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:23.725830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:23.725830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:23.725830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:23.725830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:23.725830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:23.725830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:23.727305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:23.727305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:23.727305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:23.727305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:13:23.727305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:23.727305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:23.727305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:23.727305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:23.727305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:23.727305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:23.727305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:23.727305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:23.727305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:23.727305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:23.727305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:23.727305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:23.727305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:23.727305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:23.727305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:23.727918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:23.727918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:23.727918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:23.727918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:23.727918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.727918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:23.737202 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:23.737737 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:23.738003 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:23.738172 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:13:23.738641 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:23.739289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.739891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:23.739891 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:23.739891 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:23.739891 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:23.739891 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:23.739891 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:23.739891 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.739891 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:23.741054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.741054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:13:23.741054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:23.741054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:23.741054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:23.741054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:23.741054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:23.741054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:23.741054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:23.741054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:23.741054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:23.741054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:13:23.741054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:23.741054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:13:23.741054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:13:23.741600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:23.741600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:13:23.741600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:23.741600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:13:23.741600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:13:23.741600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:23.741600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:13:23.741600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.741600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:13:23.742549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.742549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:23.742549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:23.742549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:23.742549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:23.742549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:13:23.742549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:13:23.742549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:13:23.742549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:23.742549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:23.742549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:23.742549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:23.742910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:23.742910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:23.742910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:23.742910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:23.742910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:23.742910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:23.742910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:23.742910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:23.742910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:23.742910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:23.742910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:23.742910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:23.742910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:23.742910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:23.744357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:23.744357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:23.744357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.744357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:23.744512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.744512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:23.744512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:23.744512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:23.744512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:23.744512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:23.744512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:23.744512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:23.744512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:23.744512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:23.744512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:23.744512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:23.744512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:23.744512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:23.744512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:23.744512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:23.745051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:13:23.745051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:13:23.745051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:13:23.745051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:23.745051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:23.745051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:23.745051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:23.745051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:23.745051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:23.745051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:23.745051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:23.745051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:23.745051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:23.745051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:23.745051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:23.745051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:23.745051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:23.745051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:23.745754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:23.745754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:23.745754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:23.745754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:23.745754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:23.745754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.745754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:23.747014 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:23.747014 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:23.747138 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:23.747312 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:23.747803 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:23.748371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.749063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:23.749063 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:23.749063 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:23.749063 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:23.749063 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:23.749063 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:23.749063 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.749063 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:23.752117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.752117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:13:23.752117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:23.752117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:23.752117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:23.752117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:23.752117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:23.752117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:23.752117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:23.752117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:23.752117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:23.752117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:13:23.752117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:23.752117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:13:23.752117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:13:23.752603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:23.752603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:13:23.752603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:23.752603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:13:23.752603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:13:23.752603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:23.752603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:13:23.752603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.752603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:13:23.752603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.752603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:23.752603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:23.752603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:23.752603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:23.753054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:13:23.753054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:13:23.753054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:23.753054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:23.753054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:23.753054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:23.753054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:23.753054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:23.753054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:23.753054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:23.753054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:23.753054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:23.753054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:23.753403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:23.753403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:23.753403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:23.753403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:23.753403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:23.753403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:23.753403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:23.753403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:23.753403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:23.753403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.753403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:23.753403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.753765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:23.753765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:23.753765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:23.753765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:23.753765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:23.753765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:23.753765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:23.753765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:23.753765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:23.753765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:23.753765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:23.753765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:23.753765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:23.753765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:23.753765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:23.753765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:13:23.753765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:13:23.753765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:23.753765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:23.754304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:23.754304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:23.754304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:23.754304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:23.754304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:23.754304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:23.754304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:23.754304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:23.754304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:23.754304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:23.754304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:23.754304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:23.754304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:23.754304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:23.754304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:23.754304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:23.754304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:23.754304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:23.754304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.754304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:23.754702 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:23.754702 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:23.754702 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:23.754702 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:23.754702 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:23.754823 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:23.754909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.755386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:23.755386 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:23.755386 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:23.755386 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:23.755386 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:23.755386 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:23.755386 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.755386 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:23.756978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.756978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:23.756978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:23.756978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:23.756978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:23.756978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:13:23.756978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:23.756978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:23.756978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:23.756978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:23.756978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:23.756978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:23.757276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:23.757276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:23.757276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:23.757276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:23.757276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:23.757276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:23.757276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:23.757276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:23.757276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:23.757276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:23.757276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:23.757276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:23.757276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:23.757624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.757624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:23.757624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.757624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:23.757624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:23.757624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:23.757624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:23.757624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:23.757624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:23.757624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:23.757624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:23.757624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:23.757624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:23.757624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:23.757624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:23.757994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:23.757994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:23.757994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:23.757994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:13:23.757994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:23.757994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:23.757994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:23.757994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:23.757994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:23.757994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:23.757994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:23.757994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:23.757994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:23.757994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:23.757994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:23.757994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:23.757994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:23.757994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:23.757994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:23.758487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:23.758487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:23.758487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:23.758487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:23.758487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:23.758487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:27.942887 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:27.943670 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:27.943670 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:27.944701 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:27.983576 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:27.984020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:27.984924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:27.984924 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:27.984924 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:27.984924 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:27.984924 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:27.984924 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:27.984924 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:27.984924 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:27.985776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:27.985776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:27.985776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:27.985776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:27.985776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:27.985776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:13:27.985776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:27.985776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:27.985776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:27.985776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:27.985776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:27.985776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:27.986213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:27.986213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:27.986213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:27.986213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:27.986213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:27.986213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:27.986213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:27.986213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:27.986213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:27.986213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:27.986213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:27.986213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:27.986213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:27.986652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:27.986652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:27.986652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:27.986652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:27.986652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:27.986652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:27.986652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:27.986652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:27.986652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:27.986652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:27.986652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:27.986652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:27.986652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:27.986652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:27.986652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:27.987199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:27.987199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:27.987199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:27.987199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:13:27.987199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:27.987199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:27.987199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:27.987199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:27.987199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:27.987199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:27.987199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:27.987199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:27.987199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:27.987199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:27.987199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:27.987199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:27.987199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:27.987199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:27.987199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:27.987760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:27.987760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:27.987760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:27.987760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:27.987760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:27.987760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:28.719113 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:28.719938 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:28.720174 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:28.721853 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:28.723016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.724231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:28.724231 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:28.724231 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:28.724231 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:28.724231 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:28.724231 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:28.724231 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.724231 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:28.725855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.725855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:28.725855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:28.725855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:28.725855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:28.725855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:13:28.725855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:28.725855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:28.725855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:28.725855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:28.725855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:28.725855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:28.726573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:28.726573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:28.726573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:28.726573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:28.726573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:28.726573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:28.726573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:28.726573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:28.726573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:28.726573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:28.726573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:28.726573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:28.726573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:28.727272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.727272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:28.727525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.727525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:28.727525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:28.727525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:28.727525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:28.727525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:28.727525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:28.727525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:28.727525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:28.727525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:28.727525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:28.727525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:28.727525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:28.727525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:28.727525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:28.727525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:28.728447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:13:28.728447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:28.728447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:28.728447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:28.728447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:28.728447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:28.728447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:28.728447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:28.728447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:28.728447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:28.728447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:28.728447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:28.728447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:28.728447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:28.728447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:28.728447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:28.728447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:28.728447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:28.728447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:28.729063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:28.729063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.729063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:28.737864 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:28.738268 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:28.738387 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:28.738649 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:13:28.739612 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:28.739997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.740906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:28.740906 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:28.740906 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:28.740906 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:28.740906 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:28.740906 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:28.740906 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.740906 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:28.741513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.741513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:13:28.741513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:28.741513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:28.741513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:28.741513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:28.741513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:28.741513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:28.741513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:28.741513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:28.741513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:28.741513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:13:28.741513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:28.741513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:13:28.741513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:13:28.742472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:28.742472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:13:28.742472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:28.742472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:13:28.742472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:13:28.742472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:28.742472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:13:28.742472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.742472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:13:28.742472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.742472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:28.742472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:28.742472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:28.742472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:28.743080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:13:28.743080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:13:28.743080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:13:28.743080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:28.743080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:28.743080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:28.743080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:28.743080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:28.743080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:28.743080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:28.743080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:28.743080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:28.743080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:28.743643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:28.743643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:28.743643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:28.743643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:28.743643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:28.743643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:28.743643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:28.743643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:28.743643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:28.743643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:28.743643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.743643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:28.744189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.744189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:28.744189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:28.744189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:28.744189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:28.744189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:28.744189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:28.744189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:28.744189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:28.744189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:28.744189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:28.744189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:28.744189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:28.744189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:28.744189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:28.744189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:28.744904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:13:28.744904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:13:28.744904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:13:28.744904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:28.744904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:28.744904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:28.744904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:28.744904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:28.744904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:28.744904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:28.744904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:28.744904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:28.744904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:28.744904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:28.744904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:28.744904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:28.744904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:28.744904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:28.745635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:28.745635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:28.745635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:28.745635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:28.745635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:28.745635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.745635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:28.747272 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:28.747604 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:28.747669 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:28.747748 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:28.748692 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:28.749029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.749951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:28.749951 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:28.749951 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:28.749951 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:28.749951 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:28.749951 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:28.749951 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.749951 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:28.750916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.750916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:13:28.750916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:28.750916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:28.750916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:28.750916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:28.750916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:28.750916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:28.750916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:28.750916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:28.750916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:28.750916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:13:28.750916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:28.750916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:13:28.750916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:13:28.751315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:28.751315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:13:28.751315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:28.751315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:13:28.751315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:13:28.751315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:28.751315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:13:28.751315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.751315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:13:28.752333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.752333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:28.752333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:28.752333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:28.752333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:28.752333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:13:28.752333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:13:28.752333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:28.752333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:28.752333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:28.752333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:28.752727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:28.752727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:28.752727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:28.752727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:28.752727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:28.752727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:28.752727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:28.752727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:28.752727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:28.752727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:28.752727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:28.752727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:28.752727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:28.752727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:28.753149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:28.753149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:28.753149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.753149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:28.753888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.753888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:28.753888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:28.753888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:28.753888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:28.753888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:28.753888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:28.753888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:28.753888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:28.753888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:28.753888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:28.753888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:28.753888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:28.753888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:28.753888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:28.753888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:28.753888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:13:28.754374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:13:28.754374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:28.754374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:28.754374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:28.754374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:28.754374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:28.754374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:28.754374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:28.754374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:28.754374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:28.754374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:28.754374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:28.754374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:28.754374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:28.754374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:28.754374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:28.754374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:28.754374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:28.754374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:28.754854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:28.754854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:28.754854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.754854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:28.754854 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:28.754854 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:28.754854 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:28.755068 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:28.756125 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:28.757207 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:28.757575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.758532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:28.758532 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:28.758532 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:28.758532 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:28.758532 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:28.758532 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:28.758532 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.758532 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:28.759691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.759691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:28.759691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:28.759691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:28.759691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:28.759691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:13:28.759691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:28.759691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:28.759691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:28.759691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:28.759691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:28.759691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:28.760069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:28.760069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:28.760069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:28.760069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:28.760069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:28.760069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:28.760069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:28.760069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:28.760069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:28.760069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:28.760069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:28.760069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:28.760069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:28.760400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.760400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:28.761109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.761109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:28.761109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:28.761109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:28.761109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:28.761109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:28.761109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:28.761109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:28.761109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:28.761109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:28.761109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:28.761109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:28.761109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:28.761109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:28.761109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:28.761109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:28.761547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:13:28.761547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:28.761547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:28.761547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:28.761547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:28.761547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:28.761547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:28.761547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:28.761547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:28.761547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:28.761547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:28.761547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:28.761547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:28.761547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:28.761547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:28.761547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:28.761547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:28.761547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:28.761547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:28.762047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:28.762047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:28.762047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:32.943498 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:32.944500 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:32.944817 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:32.946313 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:32.985268 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:32.985463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:32.985997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:32.985997 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:32.985997 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:32.985997 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:32.985997 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:32.985997 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:32.985997 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:32.985997 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:32.986697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:32.986697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:32.986697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:32.986697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:32.986697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:32.986697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:13:32.986697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:32.986697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:32.986697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:32.986697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:32.986697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:32.986697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:32.987279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:32.987279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:32.987279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:32.987279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:32.987279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:32.987279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:32.987279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:32.987279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:32.987279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:32.987279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:32.987279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:32.987279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:32.987279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:32.987871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:32.987871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:32.987871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:32.987871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:32.987871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:32.987871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:32.987871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:32.987871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:32.987871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:32.987871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:32.987871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:32.987871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:32.987871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:32.987871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:32.987871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:32.988627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:32.988627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:32.988627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:32.988627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:13:32.988627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:32.988627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:32.988627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:32.988627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:32.988627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:32.988627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:32.988627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:32.988627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:32.988627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:32.988627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:32.988627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:32.988627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:32.988627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:32.988627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:32.988627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:32.989462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:32.989462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:32.989462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:32.989462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:32.989462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:32.989462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:33.722238 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:33.722238 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:33.722238 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:33.722827 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:33.722987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.723456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:33.723456 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:33.723456 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:33.723456 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:33.723456 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:33.723456 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:33.723456 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.723456 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:33.724410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.724410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:33.724410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:33.724410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:33.724410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:33.724410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:13:33.724410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:33.724410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:33.724410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:33.724410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:33.724410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:33.724410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:33.724809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:33.724809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:33.724809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:33.724809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:33.724809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:33.724809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:33.724809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:33.724809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:33.724809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:33.724809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:33.724809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:33.724809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:33.724809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:33.725241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.725241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:33.725521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.725521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:33.725521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:33.725521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:33.725521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:33.725521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:33.725521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:33.725521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:33.725521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:33.725521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:33.725521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:33.725521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:33.725521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:33.725521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:33.725521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:33.725521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:33.725986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:13:33.725986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:33.725986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:33.725986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:33.725986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:33.725986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:33.725986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:33.725986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:33.725986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:33.725986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:33.725986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:33.725986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:33.725986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:33.725986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:33.725986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:33.725986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:33.725986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:33.725986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:33.725986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:33.726579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:33.726579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.726579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:33.738428 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:33.738950 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:33.739185 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:33.739429 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:13:33.739987 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:33.740318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.740970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:33.740970 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:33.740970 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:33.740970 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:33.740970 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:33.740970 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:33.740970 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.740970 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:33.741972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.741972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:13:33.741972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:33.741972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:33.741972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:33.741972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:33.741972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:33.741972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:33.741972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:33.741972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:33.741972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:33.741972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:13:33.741972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:33.741972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:13:33.741972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:13:33.742816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:33.742816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:13:33.742816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:33.742816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:13:33.742816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:13:33.742816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:33.742816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:13:33.742816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.742816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:13:33.742816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.742816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:33.742816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:33.742816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:33.742816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:33.743471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:13:33.743471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:13:33.743471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:13:33.743471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:33.743471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:33.743471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:33.743471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:33.743471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:33.743471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:33.743471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:33.743471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:33.743471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:33.743471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:33.744081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:33.744081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:33.744081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:33.744081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:33.744081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:33.744081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:33.744081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:33.744081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:33.744081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:33.744081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:33.744081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.744081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:33.744750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.744750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:33.744750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:33.744750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:33.744750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:33.744750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:33.744750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:33.744750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:33.744750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:33.744750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:33.744750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:33.744750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:33.744750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:33.744750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:33.744750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:33.744750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:33.746223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:13:33.746223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:13:33.746223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:13:33.746223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:33.746223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:33.746223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:33.746223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:33.746223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:33.746223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:33.746223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:33.746223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:33.746223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:33.746223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:33.746223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:33.746223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:33.746223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:33.746223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:33.746223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:33.747085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:33.747085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:33.747085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:33.747085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:33.747085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:33.747085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.747085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:33.747793 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:33.748070 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:33.748216 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:33.748318 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:33.748963 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:33.748963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.749462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:33.749462 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:33.749462 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:33.749462 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:33.749462 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:33.749462 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:33.749462 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.749462 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:33.750185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.750185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:13:33.750185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:33.750185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:33.750185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:33.750185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:33.750185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:33.750185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:33.750185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:33.750185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:33.750185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:33.750185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:13:33.750185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:33.750185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:13:33.750185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:13:33.750625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:33.750625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:13:33.750625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:33.750625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:13:33.750625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:13:33.750625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:33.750625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:13:33.750625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.750625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:13:33.751052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.751052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:33.751052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:33.751052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:33.751052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:33.751052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:13:33.751052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:13:33.751052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:33.751052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:33.751052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:33.751052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:33.751398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:33.751398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:33.751398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:33.751398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:33.751398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:33.751398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:33.751398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:33.751398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:33.751398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:33.751398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:33.751398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:33.751398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:33.751398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:33.751398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:33.751806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:33.751806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:33.751806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.751806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:33.752107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.752107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:33.752107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:33.752107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:33.752107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:33.752107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:33.752107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:33.752107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:33.752107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:33.752107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:33.752107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:33.752107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:33.752107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:33.752107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:33.752107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:33.752107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:33.752107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:13:33.752730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:13:33.752730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:33.752730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:33.752730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:33.752730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:33.752730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:33.752730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:33.752730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:33.752730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:33.752730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:33.752730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:33.752730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:33.752730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:33.752730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:33.752730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:33.752730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:33.752730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:33.752730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:33.752730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:33.753272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:33.753272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:33.753272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.753272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:33.754756 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:33.755049 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:33.755155 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:33.755310 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:33.755594 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:33.755874 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:33.756005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.756502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:33.756502 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:33.756502 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:33.756502 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:33.756502 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:33.756502 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:33.756502 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.756502 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:33.757373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.757373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:33.757373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:33.757373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:33.757373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:33.757373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:13:33.757373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:33.757373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:33.757373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:33.757373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:33.757373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:33.757373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:33.757745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:33.757745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:33.757745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:33.757745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:33.757745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:33.757745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:33.757745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:33.757745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:33.757745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:33.757745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:33.757745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:33.757745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:33.757745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:33.758216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.758216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:33.758309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.758309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:33.758309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:33.758309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:33.758309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:33.758309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:33.758309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:33.758309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:33.758309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:33.758309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:33.758309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:33.758309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:33.758309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:33.758309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:33.758309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:33.758309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:33.758908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:13:33.758908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:33.758908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:33.758908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:33.758908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:33.758908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:33.758908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:33.758908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:33.758908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:33.758908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:33.758908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:33.758908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:33.758908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:33.758908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:33.758908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:33.758908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:33.758908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:33.758908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:33.758908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:33.759454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:33.759454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:33.759454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:37.944451 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:37.945548 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:37.945548 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:37.947012 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:37.979460 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:37.979898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:37.980730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:37.980730 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:37.980730 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:37.980730 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:37.980730 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:37.980730 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:37.980730 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:37.980730 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:37.981645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:37.981645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:37.981645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:37.981645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:37.981645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:37.981645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:13:37.981645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:37.981645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:37.981645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:37.981645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:37.981645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:37.981645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:37.982179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:37.982179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:37.982179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:37.982179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:37.982179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:37.982179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:37.982179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:37.982179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:37.982179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:37.982179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:37.982179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:37.982179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:37.982179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:37.982694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:37.982694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:37.983059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:37.983059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:37.983059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:37.983059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:37.983059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:37.983059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:37.983059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:37.983059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:37.983059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:37.983059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:37.983059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:37.983059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:37.983059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:37.983059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:37.983059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:37.983059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:37.983997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:13:37.983997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:37.983997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:37.983997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:37.983997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:37.983997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:37.983997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:37.983997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:37.983997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:37.983997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:37.983997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:37.983997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:37.983997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:37.983997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:37.983997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:37.983997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:37.983997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:37.983997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:37.983997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:37.984761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:37.984761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:37.984761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:38.722221 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:38.722715 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:38.722863 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:38.723557 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:38.723898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.724622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:38.724622 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:38.724622 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:38.724622 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:38.724622 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:38.724622 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:38.724622 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.724622 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:38.725561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.725561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:38.725561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:38.725561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:38.725561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:38.725561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:13:38.725561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:38.725561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:38.725561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:38.725561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:38.725561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:38.725561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:38.725964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:38.725964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:38.725964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:38.725964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:38.725964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:38.725964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:38.725964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:38.725964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:38.725964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:38.725964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:38.725964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:38.725964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:38.725964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:38.726293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.726293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:38.726646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.726646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:38.726646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:38.726646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:38.726646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:38.726646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:38.726646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:38.726646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:38.726646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:38.726646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:38.726646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:38.726646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:38.726646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:38.726646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:38.726646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:38.726646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:38.727114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:13:38.727114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:38.727114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:38.727114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:38.727114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:38.727114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:38.727114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:38.727114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:38.727114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:38.727114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:38.727114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:38.727114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:38.727114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:38.727114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:38.727114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:38.727114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:38.727114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:38.727114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:38.727114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:38.727602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:38.727602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.727602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:38.739003 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:38.770475 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:38.770475 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:38.772098 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:38.772181 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:38.772266 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:13:38.772711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:38.772864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.773325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:38.773325 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:38.773325 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:38.773325 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:38.773325 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:38.773325 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:38.773325 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.773325 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:38.774331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.774331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:13:38.774331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:38.774331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:38.774331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:38.774331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:38.774331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:38.774331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:38.774331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:38.774331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:38.774331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:38.774331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:13:38.774331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:38.774331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:13:38.774331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:13:38.774631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:38.774631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:13:38.774631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:38.774631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:13:38.774631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:13:38.774631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:38.774631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:13:38.774631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.774631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:13:38.774920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.774920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:38.774920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:38.774920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:38.774920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:38.774920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:13:38.774920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:13:38.774920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:13:38.774920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:38.774920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:38.774920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:38.774920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:38.775159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:38.775159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:38.775159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:38.775159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:38.775159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:38.775159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:38.775159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:38.775159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:38.775159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:38.775159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:38.775159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:38.775159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:38.775159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:38.775159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:38.775437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:38.775437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:38.775437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.775437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:38.775620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.775620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:38.775620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:38.775620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:38.775620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:38.775620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:38.775620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:38.775620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:38.775620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:38.775620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:38.775620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:38.775620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:38.775620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:38.775620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:38.775620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:38.775620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:38.775952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:13:38.775952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:13:38.775952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:13:38.775952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:38.775952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:38.775952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:38.775952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:38.775952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:38.775952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:38.775952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:38.775952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:38.775952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:38.775952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:38.775952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:38.775952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:38.775952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:38.775952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:38.775952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:38.776297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:38.776297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:38.776297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:38.776297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:38.776297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:38.776297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.776297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:38.776297 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:38.776297 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:38.776297 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:38.776522 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:38.776522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.776664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:38.776664 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:38.776664 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:38.776664 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:38.776664 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:38.776664 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:38.776664 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.776664 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:38.777179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.777179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:13:38.777179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:38.777179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:38.777179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:38.777179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:38.777179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:38.777179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:38.777179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:38.777179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:38.777179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:38.777179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:13:38.777179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:38.777179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:13:38.777179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:13:38.777536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:38.777536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:13:38.777536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:38.777536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:13:38.777536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:13:38.777536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:38.777536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:13:38.777536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.777536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:13:38.777846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.777846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:38.777846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:38.777846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:38.777846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:38.777846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:13:38.777846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:13:38.777846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:38.777846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:38.777846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:38.777846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:38.778194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:38.778194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:38.778194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:38.778194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:38.778194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:38.778194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:38.778194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:38.778194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:38.778194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:38.778194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:38.778194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:38.778194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:38.778194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:38.778194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:38.778607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:38.778607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:38.778607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.778607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:38.778607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.778607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:38.778607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:38.778607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:38.778607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:38.778607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:38.778607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:38.778607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:38.778607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:38.778607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:38.779062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:38.779062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:38.779062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:38.779062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:38.779062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:38.779062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:38.779062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:13:38.779062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:13:38.779062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:38.779062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:38.779062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:38.779062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:38.779062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:38.779062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:38.779062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:38.779062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:38.779062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:38.779062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:38.779062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:38.779563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:38.779563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:38.779563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:38.779563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:38.779563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:38.779563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:38.779563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:38.779563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:38.779563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:38.779563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.779563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:38.779563 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:38.779563 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:38.779964 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:38.779964 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:38.779964 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:38.779964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.780143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:38.780143 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:38.780143 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:38.780143 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:38.780143 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:38.780143 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:38.780143 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.780143 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:38.780703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.780703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:38.780703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:38.780703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:38.780703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:38.780703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:13:38.780703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:38.780703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:38.780703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:38.780703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:38.780703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:38.780703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:38.781108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:38.781108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:38.781108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:38.781108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:38.781108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:38.781108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:38.781108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:38.781108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:38.781108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:38.781108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:38.781108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:38.781108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:38.781108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:38.781476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.781476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:38.781476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.781476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:38.781476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:38.781476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:38.781476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:38.781476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:38.781476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:38.781476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:38.781476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:38.781476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:38.781476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:38.781476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:38.781476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:38.781933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:38.781933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:38.781933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:38.781933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:13:38.781933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:38.781933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:38.781933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:38.781933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:38.781933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:38.781933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:38.781933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:38.781933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:38.781933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:38.781933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:38.781933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:38.781933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:38.781933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:38.781933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:38.781933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:38.782425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:38.782425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:38.782425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:38.782425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:38.782425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:38.782425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:42.945561 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:42.946572 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:42.946572 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:42.947725 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:42.983394 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:42.983842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:42.984868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:42.984868 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:42.984868 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:42.984868 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:42.984868 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:42.984868 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:42.984868 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:42.984868 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:42.985869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:42.985869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:42.985869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:42.985869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:42.985869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:42.985869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:13:42.985869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:42.985869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:42.985869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:42.985869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:42.985869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:42.985869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:42.986451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:42.986451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:42.986451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:42.986451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:42.986451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:42.986451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:42.986451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:42.986451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:42.986451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:42.986451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:42.986451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:42.986451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:42.986451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:42.987178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:42.987178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:42.987178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:42.987178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:42.987178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:42.987178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:42.987178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:42.987178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:42.987178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:42.987178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:42.987178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:42.987178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:42.987178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:42.987178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:42.987178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:42.987952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:42.987952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:42.987952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:42.987952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:13:42.987952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:42.987952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:42.987952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:42.987952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:42.987952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:42.987952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:42.987952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:42.987952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:42.987952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:42.987952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:42.987952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:42.987952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:42.987952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:42.987952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:42.987952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:42.990034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:42.990034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:42.990034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:42.990034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:42.990034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:42.990034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:43.723114 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:43.723912 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:43.724109 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:43.725258 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:43.725709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.726637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:43.726637 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:43.726637 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:43.726637 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:43.726637 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:43.726637 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:43.726637 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.726637 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:43.728488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.728488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:43.728488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:43.728488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:43.728488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:43.728488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:13:43.728488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:43.728488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:43.728488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:43.728488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:43.728488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:43.728488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:43.729997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:43.729997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:43.729997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:43.729997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:43.729997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:43.729997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:43.729997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:43.729997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:43.729997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:43.729997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:43.729997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:43.729997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:43.729997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:43.730751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.730751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:43.730751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.730751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:43.730751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:43.730751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:43.730751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:43.730751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:43.730751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:43.730751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:43.730751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:43.730751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:43.730751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:43.730751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:43.730751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:43.731405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:43.731405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:43.731405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:43.731405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:13:43.731405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:43.731405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:43.731405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:43.731405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:43.731405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:43.731405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:43.731405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:43.731405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:43.731405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:43.731405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:43.731405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:43.731405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:43.731405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:43.731405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:43.731405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:43.733410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:43.733410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:43.733410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:43.733410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:43.733410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.733410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:43.772458 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:43.773007 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:43.773179 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:43.773385 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:13:43.773987 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:43.774306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.775179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:43.775179 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:43.775179 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:43.775179 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:43.775179 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:43.775179 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:43.775179 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.775179 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:43.776379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.776379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:13:43.776379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:43.776379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:43.776379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:43.776379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:43.776379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:43.776379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:43.776379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:43.776379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:43.776379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:43.776379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:13:43.776379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:43.776379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:13:43.776379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:13:43.777014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:43.777014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:13:43.777014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:43.777014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:13:43.777014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:13:43.777014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:43.777014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:13:43.777014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.777014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:13:43.778418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.778418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:43.778418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:43.778418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:43.778418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:43.778418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:13:43.778418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:13:43.778418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:13:43.778418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:43.778418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:43.778418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:43.778418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:43.778753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:43.778753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:43.778753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:43.778753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:43.778753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:43.778753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:43.778753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:43.778753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:43.778753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:43.778753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:43.778753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:43.778753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:43.778753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:43.778753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:43.779083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:43.779083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:43.779083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.779083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:43.779083 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:43.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:43.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:43.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:43.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:43.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:43.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:43.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:43.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:43.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:43.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:43.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:43.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:43.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:43.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:43.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:43.781391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:13:43.781391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:13:43.781391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:13:43.781391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:43.781391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:43.781391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:43.781391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:43.781391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:43.781391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:43.781391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:43.781391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:43.781391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:43.781391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:43.781391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:43.781391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:43.781391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:43.781391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:43.781391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:43.781861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:43.781861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:43.781861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:43.781861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:43.781861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:43.781861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.781861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:43.781861 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:43.781861 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:43.782085 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:43.782258 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:43.782778 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:43.783160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.783989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:43.783989 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:43.783989 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:43.783989 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:43.783989 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:43.783989 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:43.783989 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.783989 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:43.784841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.784841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:13:43.784841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:43.784841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:43.784841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:43.784841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:43.784841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:43.784841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:43.784841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:43.784841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:43.784841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:43.784841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:13:43.784841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:43.784841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:13:43.784841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:13:43.785704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:43.785704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:13:43.785704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:43.785704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:13:43.785704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:13:43.785704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:43.785704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:13:43.785704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.785704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:13:43.785704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.785704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:43.785704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:43.785704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:43.785704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:43.786472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:13:43.786472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:13:43.786472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:43.786472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:43.786472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:43.786472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:43.786472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:43.786472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:43.786472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:43.786472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:43.786472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:43.786472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:43.786472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:43.787173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:43.787173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:43.787173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:43.787173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:43.787173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:43.787173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:43.787173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:43.787173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:43.787173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:43.787173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.787173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:43.787173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.787617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:43.787617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:43.787617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:43.787617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:43.787617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:43.787617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:43.787617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:43.787617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:43.787617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:43.787617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:43.787617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:43.787617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:43.787617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:43.787617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:43.787617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:43.787617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:13:43.787617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:13:43.787617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:43.787617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:43.788088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:43.788088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:43.788088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:43.788088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:43.788088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:43.788088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:43.788088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:43.788088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:43.788088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:43.788088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:43.788088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:43.788088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:43.788088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:43.788088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:43.788088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:43.788088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:43.788088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:43.788088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:43.788088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.788088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:43.788577 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:43.788577 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:43.788577 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:43.788577 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:43.788577 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:43.788577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.789122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:43.789122 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:43.789122 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:43.789122 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:43.789122 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:43.789122 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:43.789122 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.789122 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:43.789960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.789960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:43.789960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:43.789960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:43.789960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:43.789960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:13:43.789960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:43.789960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:43.789960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:43.789960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:43.789960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:43.789960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:43.790806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:43.790806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:43.790806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:43.790806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:43.790806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:43.790806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:43.790806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:43.790806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:43.790806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:43.790806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:43.790806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:43.790806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:43.790806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:43.791571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.791571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:43.791571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.791571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:43.791571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:43.791571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:43.791571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:43.791571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:43.791571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:43.791571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:43.791571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:43.791571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:43.791571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:43.791571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:43.791571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:43.792350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:43.792350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:43.792350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:43.792350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:13:43.792350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:43.792350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:43.792350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:43.792350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:43.792350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:43.792350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:43.792350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:43.792350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:43.792350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:43.792350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:43.792350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:43.792350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:43.792350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:43.792350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:43.792350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:43.792948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:43.792948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:43.792948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:43.792948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:43.792948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:43.792948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:47.946355 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:47.947069 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:47.947271 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:47.948604 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:47.978499 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:47.979032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:47.980200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:47.980200 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:47.980200 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:47.980200 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:47.980200 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:47.980200 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:47.980200 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:47.980200 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:47.981343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:47.981343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:47.981343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:47.981343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:47.981343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:47.981343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:13:47.981343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:47.981343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:47.981343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:47.981343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:47.981343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:47.981343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:47.981984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:47.981984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:47.981984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:47.981984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:47.981984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:47.981984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:47.981984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:47.981984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:47.981984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:47.981984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:47.981984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:47.981984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:47.981984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:47.982551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:47.982551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:47.982551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:47.982551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:47.982551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:47.982551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:47.982551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:47.982551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:47.982551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:47.982551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:47.982551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:47.982551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:47.982551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:47.982551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:47.982551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:47.983291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:47.983291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:47.983291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:47.983291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:13:47.983291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:47.983291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:47.983291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:47.983291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:47.983291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:47.983291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:47.983291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:47.983291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:47.983291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:47.983291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:47.983291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:47.983291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:47.983291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:47.983291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:47.983291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:47.984150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:47.984150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:47.984150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:47.984150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:47.984150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:47.984150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:48.723873 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:48.724482 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:48.724699 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:48.725869 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:48.726203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.727087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:48.727087 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:48.727087 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:48.727087 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:48.727087 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:48.727087 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:48.727087 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.727087 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:48.728792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.728792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:48.728792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:48.728792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:48.728792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:48.728792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:13:48.728792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:48.728792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:48.728792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:48.728792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:48.728792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:48.728792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:48.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:48.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:48.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:48.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:48.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:48.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:48.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:48.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:48.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:48.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:48.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:48.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:48.729409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:48.729938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.729938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:48.730831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.730831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:48.730831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:48.730831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:48.730831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:48.730831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:48.730831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:48.730831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:48.730831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:48.730831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:48.730831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:48.730831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:48.730831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:48.730831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:48.730831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:48.730831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:48.732291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:13:48.732291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:48.732291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:48.732291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:48.732291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:48.732291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:48.732291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:48.732291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:48.732291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:48.732291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:48.732291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:48.732291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:48.732291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:48.732291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:48.732291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:48.732291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:48.732291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:48.732291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:48.732291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:48.733369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:48.733369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.733369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:48.773083 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:48.773571 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:48.773808 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:48.774018 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:13:48.774581 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:48.774935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.775798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:48.775798 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:48.775798 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:48.775798 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:48.775798 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:48.775798 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:48.775798 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.775798 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:48.777276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.777276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:13:48.777276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:48.777276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:48.777276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:48.777276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:48.777276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:48.777276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:48.777276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:48.777276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:48.777276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:48.777276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:13:48.777276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:48.777276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:13:48.777276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:13:48.777955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:48.777955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:13:48.777955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:48.777955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:13:48.777955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:13:48.777955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:48.777955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:13:48.777955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.777955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:13:48.778703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.778703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:48.778703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:48.778703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:48.778703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:48.778703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:13:48.778703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:13:48.778703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:13:48.778703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:48.778703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:48.778703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:48.778703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:48.779292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:48.779292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:48.779292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:48.779292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:48.779292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:48.779292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:48.779292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:48.779292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:48.779292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:48.779292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:48.779292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:48.779292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:48.779292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:48.779292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:48.779882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:48.779882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:48.779882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.779882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:48.780512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.780512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:48.780512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:48.780512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:48.780512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:48.780512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:48.780512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:48.780512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:48.780512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:48.780512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:48.780512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:48.780512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:48.780512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:48.780512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:48.780512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:48.780512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:48.781241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:13:48.781241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:13:48.781241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:13:48.781241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:48.781241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:48.781241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:48.781241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:48.781241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:48.781241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:48.781241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:48.781241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:48.781241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:48.781241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:48.781241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:48.781241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:48.781241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:48.781241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:48.781241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:48.781730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:48.781730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:48.781730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:48.781730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:48.781730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:48.781730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.781730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:48.781913 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:48.782140 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:48.782252 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:48.782411 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:48.782919 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:48.783291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.784302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:48.784302 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:48.784302 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:48.784302 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:48.784302 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:48.784302 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:48.784302 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.784302 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:48.784804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.784804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:13:48.784804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:48.784804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:48.784804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:48.784804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:48.784804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:48.784804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:48.784804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:48.784804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:48.784804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:48.784804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:13:48.784804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:48.784804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:13:48.784804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:13:48.785212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:48.785212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:13:48.785212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:48.785212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:13:48.785212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:13:48.785212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:48.785212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:13:48.785212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.785212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:13:48.785726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.785726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:48.785726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:48.785726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:48.785726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:48.785726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:13:48.785726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:13:48.785726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:48.785726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:48.785726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:48.785726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:48.786054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:48.786054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:48.786054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:48.786054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:48.786054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:48.786054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:48.786054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:48.786054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:48.786054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:48.786054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:48.786054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:48.786054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:48.786054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:48.786054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:48.786511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:48.786511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:48.786511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.786511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:48.786797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.786797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:48.786797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:48.786797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:48.786797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:48.786797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:48.786797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:48.786797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:48.786797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:48.786797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:48.786797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:48.786797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:48.786797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:48.786797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:48.786797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:48.786797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:48.786797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:13:48.787260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:13:48.787260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:48.787260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:48.787260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:48.787260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:48.787260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:48.787260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:48.787260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:48.787260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:48.787260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:48.787260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:48.787260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:48.787260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:48.787260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:48.787260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:48.787260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:48.787260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:48.787260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:48.787260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:48.787755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:48.787755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:48.787755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.787755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:48.787755 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:48.787755 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:48.787755 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:48.787755 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:48.788001 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:48.788172 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:48.788362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.789130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:48.789130 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:48.789130 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:48.789130 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:48.789130 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:48.789130 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:48.789130 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.789130 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:48.789981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.789981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:48.789981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:48.789981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:48.789981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:48.789981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:13:48.789981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:48.789981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:48.789981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:48.789981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:48.789981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:48.789981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:48.790328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:48.790328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:48.790328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:48.790328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:48.790328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:48.790328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:48.790328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:48.790328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:48.790328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:48.790328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:48.790328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:48.790328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:48.790328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:48.790663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.790663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:48.790966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.790966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:48.790966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:48.790966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:48.790966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:48.790966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:48.790966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:48.790966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:48.790966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:48.790966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:48.790966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:48.790966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:48.790966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:48.790966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:48.790966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:48.790966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:48.791411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:13:48.791411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:48.791411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:48.791411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:48.791411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:48.791411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:48.791411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:48.791411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:48.791411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:48.791411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:48.791411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:48.791411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:48.791411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:48.791411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:48.791411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:48.791411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:48.791411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:48.791411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:48.791411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:48.791950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:48.791950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:48.791950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:52.947335 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:52.948360 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:52.948360 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:52.949731 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:52.986221 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:52.986687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:52.987707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:52.987707 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:52.987707 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:52.987707 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:52.987707 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:52.987707 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:52.987707 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:52.987707 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:52.988854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:52.988854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:52.988854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:52.988854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:52.988854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:52.988854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:13:52.988854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:52.988854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:52.988854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:52.988854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:52.988854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:52.988854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:52.989802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:52.989802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:52.989802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:52.989802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:52.989802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:52.989802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:52.989802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:52.989802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:52.989802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:52.989802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:52.989802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:52.989802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:52.989802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:52.990501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:52.990501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:52.990501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:52.990501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:52.990501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:52.990501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:52.990501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:52.990501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:52.990501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:52.990501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:52.990501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:52.990501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:52.990501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:52.990501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:52.990501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:52.991299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:52.991299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:52.991299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:52.991299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:13:52.991299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:52.991299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:52.991299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:52.991299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:52.991299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:52.991299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:52.991299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:52.991299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:52.991299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:52.991299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:52.991299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:52.991299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:52.991299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:52.991299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:52.991299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:52.992335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:52.992335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:52.992335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:52.992335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:52.992335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:52.992335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:53.724583 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:53.725468 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:53.725669 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:53.726777 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:53.727175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.728224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:53.728224 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:53.728224 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:53.728224 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:53.728224 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:53.728224 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:53.728224 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.728224 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:53.730023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.730023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:53.730023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:53.730023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:53.730023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:53.730023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:13:53.730023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:53.730023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:53.730023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:53.730023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:53.730023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:53.730023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:53.730578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:53.730578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:53.730578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:53.730578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:53.730578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:53.730578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:53.730578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:53.730578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:53.730578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:53.730578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:53.730578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:53.730578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:53.730578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:53.731169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.731169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:53.731857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.731857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:53.731857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:53.731857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:53.731857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:53.731857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:53.731857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:53.731857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:53.731857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:53.731857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:53.731857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:53.731857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:53.731857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:53.731857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:53.731857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:53.731857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:53.732541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:13:53.732541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:53.732541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:53.732541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:53.732541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:53.732541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:53.732541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:53.732541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:53.732541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:53.732541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:53.732541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:53.732541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:53.732541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:53.732541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:53.732541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:53.732541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:53.732541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:53.732541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:53.732541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:53.733392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:53.733392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.733392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:53.773781 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:53.774812 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:53.774909 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:53.775044 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:13:53.776349 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:53.776713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.777822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:53.777822 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:53.777822 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:53.777822 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:53.777822 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:53.777822 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:53.777822 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.777822 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:53.778922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.778922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:13:53.778922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:53.778922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:53.778922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:53.778922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:53.778922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:53.778922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:53.778922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:53.778922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:53.778922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:53.778922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:13:53.778922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:53.778922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:13:53.778922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:13:53.779580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:53.779580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:13:53.779580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:53.779580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:13:53.779580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:13:53.779580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:53.779580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:13:53.779580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.779580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:13:53.780018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.780018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:53.780018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:53.780018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:53.780018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:53.780018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:13:53.780018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:13:53.780018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:13:53.780018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:53.780018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:53.780018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:53.780018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:53.780541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:53.780541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:53.780541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:53.780541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:53.780541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:53.780541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:53.780541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:53.780541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:53.780541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:53.780541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:53.780541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:53.780541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:53.780541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:53.780541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:53.781187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:53.781187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:53.781187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.781187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:53.781473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.781473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:53.781473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:53.781473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:53.781473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:53.781473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:53.781473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:53.781473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:53.781473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:53.781473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:53.781473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:53.781473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:53.781473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:53.781473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:53.781473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:53.781473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:53.782230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:13:53.782230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:13:53.782230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:13:53.782230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:53.782230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:53.782230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:53.782230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:53.782230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:53.782230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:53.782230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:53.782230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:53.782230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:53.782230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:53.782230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:53.782230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:53.782230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:53.782230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:53.782230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:53.782901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:53.782901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:53.782901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:53.782901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:53.782901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:53.782901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.782901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:53.782901 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:53.782901 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:53.782901 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:53.783192 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:53.783373 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:53.783599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.784187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:53.784187 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:53.784187 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:53.784187 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:53.784187 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:53.784187 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:53.784187 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.784187 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:53.785052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.785052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:13:53.785052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:53.785052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:53.785052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:53.785052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:53.785052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:53.785052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:53.785052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:53.785052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:53.785052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:53.785052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:13:53.785052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:53.785052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:13:53.785052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:13:53.785754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:53.785754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:13:53.785754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:53.785754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:13:53.785754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:13:53.785754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:53.785754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:13:53.785754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.785754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:13:53.786175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.786175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:53.786175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:53.786175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:53.786175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:53.786175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:13:53.786175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:13:53.786175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:53.786175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:53.786175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:53.786175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:53.786492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:53.786492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:53.786492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:53.786492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:53.786492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:53.786492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:53.786492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:53.786492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:53.786492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:53.786492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:53.786492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:53.786492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:53.786492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:53.786492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:53.786901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:53.786901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:53.786901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.786901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:53.787246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.787246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:53.787246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:53.787246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:53.787246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:53.787246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:53.787246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:53.787246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:53.787246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:53.787246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:53.787246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:53.787246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:53.787246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:53.787246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:53.787246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:53.787246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:53.787246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:13:53.787727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:13:53.787727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:53.787727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:53.787727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:53.787727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:53.787727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:53.787727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:53.787727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:53.787727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:53.787727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:53.787727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:53.787727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:53.787727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:53.787727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:53.787727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:53.787727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:53.787727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:53.787727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:53.787727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:53.788252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:53.788252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:53.788252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.788252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:53.788252 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:53.788252 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:53.788252 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:53.788252 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:53.788582 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:53.788831 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:53.788955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.789600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:53.789600 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:53.789600 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:53.789600 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:53.789600 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:53.789600 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:53.789600 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.789600 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:53.790475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.790475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:53.790475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:53.790475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:53.790475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:53.790475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:13:53.790475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:53.790475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:53.790475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:53.790475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:53.790475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:53.790475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:53.790860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:53.790860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:53.790860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:53.790860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:53.790860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:53.790860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:53.790860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:53.790860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:53.790860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:53.790860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:53.790860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:53.790860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:53.790860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:53.791219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.791219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:53.791519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.791519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:53.791519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:53.791519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:53.791519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:53.791519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:53.791519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:53.791519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:53.791519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:53.791519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:53.791519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:53.791519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:53.791519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:53.791519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:53.791519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:53.791519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:53.791969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:13:53.791969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:53.791969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:53.791969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:53.791969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:53.791969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:53.791969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:53.791969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:53.791969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:53.791969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:53.791969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:53.791969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:53.791969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:53.791969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:53.791969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:53.791969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:53.791969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:53.791969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:53.791969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:53.792514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:53.792514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:53.792514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:57.947991 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:57.948794 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:57.948794 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:57.950013 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:57.985306 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:57.985504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:57.986152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:57.986152 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:57.986152 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:57.986152 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:57.986152 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:57.986152 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:57.986152 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:57.986152 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:57.987249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:57.987249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:57.987249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:57.987249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:57.987249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:57.987249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:13:57.987249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:57.987249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:57.987249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:57.987249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:57.987249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:57.987249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:57.987973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:57.987973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:57.987973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:57.987973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:57.987973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:57.987973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:57.987973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:57.987973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:57.987973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:57.987973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:57.987973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:57.987973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:57.987973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:57.988554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:57.988554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:57.988554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:57.988554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:57.988554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:57.988554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:57.988554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:57.988554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:57.988554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:57.988554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:57.988554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:57.988554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:57.988554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:57.988554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:57.988554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:57.989313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:57.989313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:57.989313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:57.989313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:13:57.989313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:57.989313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:57.989313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:57.989313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:57.989313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:57.989313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:57.989313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:57.989313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:57.989313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:57.989313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:57.989313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:57.989313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:57.989313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:57.989313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:57.989313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:57.990204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:57.990204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:57.990204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:57.990204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:57.990204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:57.990204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:58.725402 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:58.725976 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:58.726173 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:58.727229 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:58.727582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.728584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:58.728584 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:58.728584 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:58.728584 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:58.728584 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:58.728584 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:58.728584 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.728584 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:58.730642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.730642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:58.730642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:58.730642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:58.730642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:58.730642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:13:58.730642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:58.730642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:58.730642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:58.730642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:58.730642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:58.730642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:58.731710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:58.731710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:58.731710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:58.731710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:58.731710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:58.731710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:58.731710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:58.731710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:58.731710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:58.731710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:58.731710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:58.731710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:58.731710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:58.732352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.732352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:58.733553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.733553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:58.733553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:58.733553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:58.733553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:58.733553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:58.733553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:58.733553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:58.733553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:58.733553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:58.733553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:58.733553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:58.733553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:58.733553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:58.733553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:58.733553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:58.734507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:13:58.734507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:58.734507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:58.734507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:58.734507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:58.734507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:58.734507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:58.734507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:58.734507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:58.734507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:58.734507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:58.734507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:58.734507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:58.734507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:58.734507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:58.734507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:58.734507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:58.734507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:58.734507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:58.735105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:58.735105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.735105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:58.774592 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:58.775242 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:58.775434 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:58.775633 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:13:58.776295 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:58.776698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.777490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:58.777490 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:58.777490 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:58.777490 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:58.777490 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:58.777490 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:58.777490 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.777490 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:58.778952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.778952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:13:58.778952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:58.778952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:58.778952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:58.778952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:58.778952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:58.778952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:58.778952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:58.778952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:58.778952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:58.778952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:13:58.778952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:58.778952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:13:58.778952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:13:58.779723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:58.779723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:13:58.779723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:58.779723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:13:58.779723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:13:58.779723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:58.779723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:13:58.779723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.779723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:13:58.780593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.780593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:58.780593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:58.780593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:58.780593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:58.780593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:13:58.780593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:13:58.780593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:13:58.780593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:58.780593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:58.780593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:58.780593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:58.781278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:58.781278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:58.781278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:58.781278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:58.781278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:58.781278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:58.781278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:58.781278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:58.781278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:58.781278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:58.781278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:58.781278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:58.781278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:58.781278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:58.781914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:58.781914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:58.781914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.781914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:58.781914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.781914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:58.781914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:58.781914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:58.781914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:58.781914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:58.781914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:58.781914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:58.781914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:58.781914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:58.782577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:58.782577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:58.782577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:58.782577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:58.782577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:58.782577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:58.782577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:13:58.782577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:13:58.782577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:13:58.782577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:58.782577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:58.782577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:58.782577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:58.782577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:58.782577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:58.782577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:58.782577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:58.782577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:58.782577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:58.783525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:58.783525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:58.783525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:58.783525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:58.783525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:58.783525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:58.783525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:58.783525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:58.783525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:58.783525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:58.783525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.783525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:58.783525 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:58.783525 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:58.784344 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:58.784344 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:58.784344 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:58.784344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.785011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:58.785011 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:58.785011 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:58.785011 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:58.785011 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:58.785011 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:58.785011 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.785011 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:58.786064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.786064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:13:58.786064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:58.786064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:58.786064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:58.786064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:58.786064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:58.786064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:58.786064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:58.786064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:13:58.786064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:58.786064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:13:58.786064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:58.786064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:13:58.786064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:13:58.786446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:58.786446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:13:58.786446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:58.786446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:13:58.786446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:13:58.786446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:58.786446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:13:58.786446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.786446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:13:58.787098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.787098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:58.787098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:58.787098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:58.787098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:58.787098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:13:58.787098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:13:58.787098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:58.787098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:58.787098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:58.787098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:58.787410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:58.787410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:58.787410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:58.787410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:58.787410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:58.787410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:58.787410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:58.787410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:58.787410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:58.787410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:58.787410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:58.787410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:58.787410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:58.787410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:58.787799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:58.787799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:58.787799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.787799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:58.788435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.788435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:58.788435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:58.788435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:58.788435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:58.788435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:58.788435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:58.788435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:58.788435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:58.788435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:58.788435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:58.788435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:58.788435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:58.788435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:58.788435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:58.788435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:58.788435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:13:58.788966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:13:58.788966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:13:58.788966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:13:58.788966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:58.788966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:58.788966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:58.788966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:58.788966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:58.788966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:58.788966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:58.788966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:58.788966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:58.788966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:58.788966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:58.788966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:58.788966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:58.788966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:58.788966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:58.788966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:58.789443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:58.789443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:58.789443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.789443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:13:58.789443 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:13:58.789443 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:13:58.789443 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:13:58.789443 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:13:58.789889 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:58.790254 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:13:58.790447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.791234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:13:58.791234 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:58.791234 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:13:58.791234 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:13:58.791234 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:58.791234 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:13:58.791234 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.791234 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:13:58.792253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.792253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:13:58.792253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:58.792253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:13:58.792253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:58.792253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:13:58.792253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:58.792253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:58.792253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:58.792253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:58.792253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:58.792253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:58.792610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:13:58.792610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:58.792610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:13:58.792610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:58.792610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:13:58.792610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:13:58.792610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:58.792610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:13:58.792610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:58.792610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:13:58.792610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:13:58.792610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:58.792610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:13:58.792967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.792967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:13:58.793455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.793455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:13:58.793455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:13:58.793455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:13:58.793455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:13:58.793455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:13:58.793455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:58.793455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:13:58.793455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:13:58.793455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:58.793455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:13:58.793455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:13:58.793455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:13:58.793455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:13:58.793455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:13:58.793455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:13:58.793910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:13:58.793910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:13:58.793910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:58.793910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:58.793910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:13:58.793910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:13:58.793910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:13:58.793910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:13:58.793910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:13:58.793910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:13:58.793910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:13:58.793910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:13:58.793910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:13:58.793910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:13:58.793910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:13:58.793910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:13:58.793910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:13:58.793910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:13:58.793910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:13:58.794393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:13:58.794393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:13:58.794393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:02.948608 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:02.949490 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:02.949490 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:02.950101 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:02.968896 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=233 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:14:02.988604 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:02.989022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:02.989876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:02.989876 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:02.989876 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:02.989876 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:02.989876 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:02.989876 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:02.989876 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:02.989876 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:02.990880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:02.990880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:02.990880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:02.990880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:02.990880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:02.990880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:14:02.990880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:02.990880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:02.990880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:02.990880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:02.990880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:02.990880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:02.991439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:02.991439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:02.991439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:02.991439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:02.991439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:02.991439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:02.991439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:02.991439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:02.991439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:02.991439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:02.991439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:02.991439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:02.991439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:02.992019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:02.992019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:02.992138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:02.992138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:02.992138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:02.992138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:02.992138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:02.992138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:02.992138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:02.992138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:02.992138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:02.992138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:02.992138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:02.992138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:02.992138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:02.992138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:02.992138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:02.992138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:02.993456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:14:02.993456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:02.993456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:02.993456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:02.993456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:02.993456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:02.993456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:02.993456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:02.993456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:02.993456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:02.993456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:02.993456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:02.993456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:02.993456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:02.993456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:02.993456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:02.993456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:02.993456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:02.993456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:02.994889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:02.994889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:02.994889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:03.726223 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:03.727187 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:03.727396 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:03.728589 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:03.728981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.729915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:03.729915 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:03.729915 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:03.729915 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:03.729915 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:03.729915 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:03.729915 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.729915 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:03.731617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.731617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:03.731617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:03.731617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:03.731617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:03.731617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:14:03.731617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:03.731617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:03.731617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:03.731617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:03.731617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:03.731617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:03.732263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:03.732263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:03.732263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:03.732263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:03.732263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:03.732263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:03.732263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:03.732263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:03.732263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:03.732263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:03.732263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:03.732263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:03.732263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:03.732911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.732911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:03.733543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.733543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:03.733543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:03.733543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:03.733543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:03.733543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:03.733543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:03.733543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:03.733543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:03.733543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:03.733543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:03.733543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:03.733543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:03.733543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:03.733543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:03.733543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:03.734021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:14:03.734021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:03.734021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:03.734021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:03.734021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:03.734021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:03.734021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:03.734021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:03.734021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:03.734021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:03.734021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:03.734021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:03.734021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:03.734021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:03.734021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:03.734021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:03.734021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:03.734021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:03.734021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:03.734484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:03.734484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.734484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:03.775428 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:03.775997 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:03.776182 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:03.776380 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:14:03.777250 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:03.777609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.778556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:03.778556 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:03.778556 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:03.778556 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:03.778556 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:03.778556 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:03.778556 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.778556 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:03.779958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.779958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:14:03.779958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:03.779958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:03.779958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:03.779958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:03.779958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:03.779958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:03.779958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:03.779958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:03.779958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:03.779958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:14:03.779958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:03.779958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:14:03.779958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:14:03.780569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:03.780569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:14:03.780569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:03.780569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:14:03.780569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:14:03.780569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:03.780569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:14:03.780569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.780569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:14:03.781402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.781402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:03.781402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:03.781402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:03.781402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:03.781402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:14:03.781402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:14:03.781402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:14:03.781402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:03.781402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:03.781402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:03.781402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:03.781957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:03.781957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:03.781957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:03.781957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:03.781957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:03.781957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:03.781957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:03.781957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:03.781957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:03.781957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:03.781957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:03.781957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:03.781957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:03.781957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:03.782477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:03.782477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:03.782477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.782477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:03.783304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.783304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:03.783304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:03.783304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:03.783304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:03.783304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:03.783304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:03.783304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:03.783304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:03.783304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:03.783304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:03.783304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:03.783304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:03.783304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:03.783304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:03.783304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:03.784353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:14:03.784353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:14:03.784353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:14:03.784353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:03.784353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:03.784353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:03.784353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:03.784353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:03.784353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:03.784353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:03.784353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:03.784353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:03.784353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:03.784353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:03.784353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:03.784353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:03.784353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:03.784353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:03.785325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:03.785325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:03.785325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:03.785325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:03.785325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:03.785325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.785325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:03.785325 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:03.785325 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:03.785325 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:03.785896 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:03.785896 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:03.785896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.785896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:03.785896 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:03.785896 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:03.785896 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:03.785896 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:03.785896 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:03.785896 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.785896 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:03.786532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.786532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:14:03.786532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:03.786532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:03.786532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:03.786532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:03.786532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:03.786532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:03.786532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:03.786532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:03.786532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:03.786532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:14:03.786532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:03.786532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:14:03.786532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:14:03.786961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:03.786961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:14:03.786961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:03.786961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:14:03.786961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:14:03.786961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:03.786961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:14:03.786961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.786961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:14:03.787532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.787532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:03.787532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:03.787532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:03.787532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:03.787532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:14:03.787532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:14:03.787532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:03.787532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:03.787532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:03.787532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:03.788111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:03.788111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:03.788111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:03.788111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:03.788111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:03.788111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:03.788111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:03.788111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:03.788111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:03.788111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:03.788111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:03.788111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:03.788111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:03.788111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:03.788871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:03.788871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:03.788871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.788871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:03.788871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.788871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:03.788871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:03.788871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:03.788871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:03.788871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:03.788871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:03.788871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:03.788871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:03.788871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:03.789285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:03.789285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:03.789285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:03.789285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:03.789285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:03.789285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:03.789285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:14:03.789285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:14:03.789285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:03.789285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:03.789285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:03.789285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:03.789285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:03.789285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:03.789285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:03.789285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:03.789285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:03.789285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:03.789285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:03.789786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:03.789786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:03.789786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:03.789786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:03.789786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:03.789786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:03.789786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:03.789786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:03.789786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:03.789786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.789786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:03.789786 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:03.789786 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:03.790162 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:03.790162 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:03.790162 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:03.790409 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:03.790722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.791458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:03.791458 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:03.791458 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:03.791458 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:03.791458 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:03.791458 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:03.791458 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.791458 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:03.792303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.792303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:03.792303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:03.792303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:03.792303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:03.792303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:14:03.792303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:03.792303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:03.792303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:03.792303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:03.792303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:03.792303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:03.792827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:03.792827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:03.792827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:03.792827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:03.792827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:03.792827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:03.792827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:03.792827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:03.792827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:03.792827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:03.792827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:03.792827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:03.792827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:03.793319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.793319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:03.793319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.793319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:03.793319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:03.793319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:03.793319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:03.793319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:03.793319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:03.793319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:03.793319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:03.793319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:03.793319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:03.793319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:03.793319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:03.793841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:03.793841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:03.793841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:03.793841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:14:03.793841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:03.793841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:03.793841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:03.793841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:03.793841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:03.793841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:03.793841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:03.793841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:03.793841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:03.793841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:03.793841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:03.793841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:03.793841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:03.793841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:03.793841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:03.794341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:03.794341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:03.794341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:03.794341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:03.794341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:03.794341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:07.949340 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:07.950316 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:07.950316 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:07.951398 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:07.986859 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:07.987242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:07.988090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:07.988090 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:07.988090 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:07.988090 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:07.988090 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:07.988090 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:07.988090 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:07.988090 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:07.988858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:07.988858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:07.988858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:07.988858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:07.988858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:07.988858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:14:07.988858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:07.988858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:07.988858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:07.988858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:07.988858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:07.988858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:07.989404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:07.989404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:07.989404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:07.989404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:07.989404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:07.989404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:07.989404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:07.989404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:07.989404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:07.989404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:07.989404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:07.989404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:07.989404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:07.990034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:07.990034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:07.990034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:07.990034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:07.990034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:07.990034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:07.990034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:07.990034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:07.990034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:07.990034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:07.990034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:07.990034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:07.990034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:07.990034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:07.990034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:07.991120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:07.991120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:07.991120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:07.991120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:14:07.991120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:07.991120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:07.991120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:07.991120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:07.991120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:07.991120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:07.991120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:07.991120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:07.991120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:07.991120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:07.991120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:07.991120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:07.991120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:07.991120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:07.991120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:07.992670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:07.992670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:07.992670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:07.992670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:07.992670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:07.992670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:08.726940 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:08.727431 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:08.727532 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:08.728313 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:08.728700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.729477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:08.729477 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:08.729477 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:08.729477 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:08.729477 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:08.729477 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:08.729477 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.729477 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:08.730442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.730442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:08.730442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:08.730442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:08.730442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:08.730442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:14:08.730442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:08.730442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:08.730442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:08.730442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:08.730442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:08.730442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:08.731468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:08.731468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:08.731468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:08.731468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:08.731468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:08.731468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:08.731468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:08.731468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:08.731468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:08.731468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:08.731468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:08.731468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:08.731468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:08.732017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.732017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:08.732017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.732017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:08.732017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:08.732017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:08.732017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:08.732017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:08.732017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:08.732017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:08.732017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:08.732017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:08.732017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:08.732017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:08.732017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:08.733204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:08.733204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:08.733204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:08.733204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:14:08.733204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:08.733204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:08.733204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:08.733204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:08.733204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:08.733204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:08.733204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:08.733204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:08.733204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:08.733204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:08.733204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:08.733204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:08.733204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:08.733204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:08.733204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:08.734819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:08.734819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:08.734819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:08.734819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:08.734819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.734819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:08.776424 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:08.777362 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:08.777362 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:08.777513 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:14:08.779000 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:08.779158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.780299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:08.780299 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:08.780299 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:08.780299 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:08.780299 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:08.780299 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:08.780299 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.780299 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:08.781927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.781927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:14:08.781927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:08.781927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:08.781927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:08.781927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:08.781927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:08.781927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:08.781927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:08.781927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:08.781927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:08.781927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:14:08.781927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:08.781927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:14:08.781927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:14:08.782417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:08.782417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:14:08.782417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:08.782417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:14:08.782417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:14:08.782417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:08.782417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:14:08.782417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.782417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:14:08.783249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.783249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:08.783249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:08.783249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:08.783249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:08.783249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:14:08.783249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:14:08.783249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:14:08.783249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:08.783249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:08.783249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:08.783249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:08.783655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:08.783655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:08.783655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:08.783655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:08.783655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:08.783655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:08.783655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:08.783655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:08.783655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:08.783655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:08.783655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:08.783655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:08.783655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:08.783655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:08.784136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:08.784136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:08.784136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.784136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:08.785038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.785038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:08.785038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:08.785038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:08.785038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:08.785038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:08.785038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:08.785038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:08.785038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:08.785038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:08.785038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:08.785038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:08.785038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:08.785038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:08.785038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:08.785038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:08.785472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:14:08.785472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:14:08.785472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:14:08.785472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:08.785472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:08.785472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:08.785472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:08.785472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:08.785472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:08.785472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:08.785472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:08.785472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:08.785472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:08.785472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:08.785472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:08.785472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:08.785472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:08.785472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:08.785940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:08.785940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:08.785940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:08.785940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:08.785940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:08.785940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.785940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:08.785940 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:08.785940 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:08.785940 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:08.786238 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:08.786238 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:08.786343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.786814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:08.786814 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:08.786814 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:08.786814 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:08.786814 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:08.786814 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:08.786814 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.786814 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:08.787324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.787324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:14:08.787324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:08.787324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:08.787324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:08.787324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:08.787324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:08.787324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:08.787324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:08.787324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:08.787324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:08.787324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:14:08.787324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:08.787324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:14:08.787324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:14:08.787821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:08.787821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:14:08.787821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:08.787821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:14:08.787821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:14:08.787821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:08.787821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:14:08.787821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.787821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:14:08.788057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.788057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:08.788057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:08.788057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:08.788057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:08.788057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:14:08.788057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:14:08.788057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:08.788057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:08.788057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:08.788057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:08.788358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:08.788358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:08.788358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:08.788358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:08.788358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:08.788358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:08.788358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:08.788358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:08.788358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:08.788358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:08.788358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:08.788358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:08.788358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:08.788358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:08.788775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:08.788775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:08.788775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.788775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:08.788775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.788775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:08.788775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:08.788775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:08.788775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:08.788775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:08.788775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:08.788775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:08.788775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:08.788775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:08.789146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:08.789146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:08.789146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:08.789146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:08.789146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:08.789146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:08.789146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:14:08.789146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:14:08.789146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:08.789146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:08.789146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:08.789146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:08.789146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:08.789146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:08.789146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:08.789146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:08.789146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:08.789146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:08.789146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:08.789638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:08.789638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:08.789638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:08.789638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:08.789638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:08.789638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:08.789638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:08.789638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:08.789638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:08.789638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.789638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:08.789638 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:08.789638 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:08.790056 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:08.790056 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:08.790139 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:08.790353 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:08.790473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.790935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:08.790935 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:08.790935 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:08.790935 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:08.790935 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:08.790935 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:08.790935 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.790935 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:08.791577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.791577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:08.791577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:08.791577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:08.791577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:08.791577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:14:08.791577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:08.791577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:08.791577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:08.791577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:08.791577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:08.791577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:08.791936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:08.791936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:08.791936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:08.791936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:08.791936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:08.791936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:08.791936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:08.791936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:08.791936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:08.791936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:08.791936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:08.791936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:08.791936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:08.792283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.792283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:08.792283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.792283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:08.792283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:08.792283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:08.792283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:08.792283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:08.792283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:08.792283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:08.792283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:08.792283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:08.792283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:08.792283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:08.792283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:08.792755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:08.792755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:08.792755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:08.792755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:14:08.792755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:08.792755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:08.792755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:08.792755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:08.792755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:08.792755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:08.792755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:08.792755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:08.792755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:08.792755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:08.792755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:08.792755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:08.792755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:08.792755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:08.792755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:08.793334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:08.793334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:08.793334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:08.793334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:08.793334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:08.793334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:12.950207 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:12.950793 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:12.950972 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:12.952221 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:12.987387 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:12.987925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:12.988891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:12.988891 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:12.988891 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:12.988891 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:12.988891 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:12.988891 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:12.988891 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:12.988891 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:12.989883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:12.989883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:12.989883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:12.989883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:12.989883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:12.989883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:14:12.989883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:12.989883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:12.989883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:12.989883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:12.989883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:12.989883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:12.990462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:12.990462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:12.990462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:12.990462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:12.990462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:12.990462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:12.990462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:12.990462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:12.990462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:12.990462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:12.990462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:12.990462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:12.990462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:12.991060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:12.991060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:12.991060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:12.991060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:12.991060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:12.991060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:12.991060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:12.991060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:12.991060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:12.991060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:12.991060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:12.991060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:12.991060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:12.991060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:12.991060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:12.992081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:12.992081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:12.992081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:12.992081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:14:12.992081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:12.992081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:12.992081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:12.992081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:12.992081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:12.992081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:12.992081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:12.992081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:12.992081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:12.992081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:12.992081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:12.992081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:12.992081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:12.992081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:12.992081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:12.992933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:12.992933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:12.992933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:12.992933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:12.992933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:12.992933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:13.727576 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:13.728287 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:13.728538 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:13.729595 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:13.730048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.731023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:13.731023 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:13.731023 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:13.731023 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:13.731023 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:13.731023 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:13.731023 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.731023 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:13.732595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.732595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:13.732595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:13.732595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:13.732595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:13.732595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:14:13.732595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:13.732595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:13.732595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:13.732595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:13.732595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:13.732595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:13.733441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:13.733441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:13.733441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:13.733441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:13.733441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:13.733441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:13.733441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:13.733441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:13.733441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:13.733441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:13.733441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:13.733441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:13.733441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:13.734053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.734053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:13.734379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.734379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:13.734379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:13.734379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:13.734379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:13.734379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:13.734379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:13.734379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:13.734379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:13.734379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:13.734379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:13.734379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:13.734379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:13.734379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:13.734379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:13.734379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:13.735141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:14:13.735141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:13.735141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:13.735141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:13.735141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:13.735141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:13.735141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:13.735141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:13.735141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:13.735141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:13.735141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:13.735141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:13.735141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:13.735141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:13.735141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:13.735141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:13.735141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:13.735141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:13.735141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:13.736064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:13.736064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.736064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:13.777268 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:13.777637 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:13.777896 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:13.778114 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:14:13.778711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:13.779061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.779983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:13.779983 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:13.779983 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:13.779983 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:13.779983 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:13.779983 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:13.779983 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.779983 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:13.781292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.781292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:14:13.781292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:13.781292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:13.781292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:13.781292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:13.781292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:13.781292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:13.781292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:13.781292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:13.781292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:13.781292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:14:13.781292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:13.781292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:14:13.781292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:14:13.782094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:13.782094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:14:13.782094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:13.782094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:14:13.782094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:14:13.782094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:13.782094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:14:13.782094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.782094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:14:13.782947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.782947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:13.782947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:13.782947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:13.782947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:13.782947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:14:13.782947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:14:13.782947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:14:13.782947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:13.782947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:13.782947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:13.782947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:13.783396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:13.783396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:13.783396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:13.783396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:13.783396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:13.783396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:13.783396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:13.783396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:13.783396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:13.783396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:13.783396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:13.783396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:13.783396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:13.783396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:13.783862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:13.783862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:13.783862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.783862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:13.784288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.784288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:13.784288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:13.784288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:13.784288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:13.784288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:13.784288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:13.784288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:13.784288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:13.784288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:13.784288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:13.784288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:13.784288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:13.784288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:13.784288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:13.784288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:13.784807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:14:13.784807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:14:13.784807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:14:13.784807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:13.784807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:13.784807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:13.784807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:13.784807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:13.784807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:13.784807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:13.784807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:13.784807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:13.784807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:13.784807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:13.784807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:13.784807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:13.784807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:13.784807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:13.785344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:13.785344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:13.785344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:13.785344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:13.785344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:13.785344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.785344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:13.785748 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:13.786009 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:13.786116 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:13.786240 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:13.786594 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:13.786896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.787431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:13.787431 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:13.787431 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:13.787431 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:13.787431 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:13.787431 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:13.787431 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.787431 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:13.788173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.788173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:14:13.788173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:13.788173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:13.788173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:13.788173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:13.788173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:13.788173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:13.788173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:13.788173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:13.788173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:13.788173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:14:13.788173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:13.788173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:14:13.788173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:14:13.788632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:13.788632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:14:13.788632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:13.788632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:14:13.788632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:14:13.788632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:13.788632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:14:13.788632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.788632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:14:13.789022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.789022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:13.789022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:13.789022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:13.789022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:13.789022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:14:13.789022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:14:13.789022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:13.789022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:13.789022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:13.789022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:13.789375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:13.789375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:13.789375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:13.789375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:13.789375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:13.789375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:13.789375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:13.789375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:13.789375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:13.789375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:13.789375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:13.789375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:13.789375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:13.789375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:13.789777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:13.789777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:13.789777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.789777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:13.790109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.790109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:13.790109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:13.790109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:13.790109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:13.790109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:13.790109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:13.790109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:13.790109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:13.790109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:13.790109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:13.790109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:13.790109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:13.790109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:13.790109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:13.790109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:13.790109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:14:13.790653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:14:13.790653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:13.790653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:13.790653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:13.790653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:13.790653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:13.790653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:13.790653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:13.790653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:13.790653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:13.790653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:13.790653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:13.790653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:13.790653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:13.790653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:13.790653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:13.790653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:13.790653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:13.790653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:13.791212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:13.791212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:13.791212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.791212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:13.791212 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:13.791212 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:13.791212 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:13.791212 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:13.791502 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:13.791502 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:13.791645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.792226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:13.792226 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:13.792226 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:13.792226 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:13.792226 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:13.792226 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:13.792226 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.792226 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:13.793116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.793116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:13.793116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:13.793116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:13.793116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:13.793116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:14:13.793116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:13.793116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:13.793116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:13.793116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:13.793116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:13.793116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:13.793498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:13.793498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:13.793498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:13.793498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:13.793498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:13.793498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:13.793498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:13.793498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:13.793498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:13.793498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:13.793498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:13.793498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:13.793498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:13.793890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.793890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:13.794151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.794151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:13.794151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:13.794151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:13.794151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:13.794151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:13.794151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:13.794151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:13.794151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:13.794151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:13.794151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:13.794151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:13.794151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:13.794151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:13.794151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:13.794151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:13.794618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:14:13.794618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:13.794618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:13.794618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:13.794618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:13.794618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:13.794618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:13.794618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:13.794618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:13.794618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:13.794618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:13.794618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:13.794618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:13.794618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:13.794618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:13.794618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:13.794618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:13.794618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:13.794618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:13.795157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:13.795157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:13.795157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:17.951052 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:17.951927 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:17.951927 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:17.953135 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:17.983666 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:17.984219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:17.985159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:17.985159 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:17.985159 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:17.985159 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:17.985159 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:17.985159 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:17.985159 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:17.985159 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:17.986772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:17.986772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:17.986772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:17.986772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:17.986772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:17.986772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:14:17.986772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:17.986772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:17.986772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:17.986772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:17.986772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:17.986772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:17.987541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:17.987541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:17.987541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:17.987541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:17.987541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:17.987541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:17.987541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:17.987541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:17.987541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:17.987541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:17.987541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:17.987541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:17.987541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:17.988195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:17.988195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:17.988312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:17.988312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:17.988312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:17.988312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:17.988312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:17.988312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:17.988312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:17.988312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:17.988312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:17.988312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:17.988312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:17.988312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:17.988312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:17.988312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:17.988312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:17.988312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:17.989079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:14:17.989079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:17.989079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:17.989079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:17.989079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:17.989079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:17.989079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:17.989079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:17.989079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:17.989079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:17.989079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:17.989079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:17.989079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:17.989079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:17.989079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:17.989079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:17.989079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:17.989079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:17.989079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:17.989954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:17.989954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:17.989954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:18.728359 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:18.728665 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:18.728846 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:18.729871 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:18.730230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.731202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:18.731202 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:18.731202 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:18.731202 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:18.731202 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:18.731202 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:18.731202 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.731202 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:18.732996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.732996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:18.732996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:18.732996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:18.732996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:18.732996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:14:18.732996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:18.732996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:18.732996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:18.732996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:18.732996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:18.732996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:18.733620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:18.733620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:18.733620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:18.733620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:18.733620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:18.733620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:18.733620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:18.733620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:18.733620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:18.733620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:18.733620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:18.733620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:18.733620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:18.734223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.734223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:18.734836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.734836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:18.734836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:18.734836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:18.734836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:18.734836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:18.734836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:18.734836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:18.734836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:18.734836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:18.734836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:18.734836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:18.734836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:18.734836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:18.734836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:18.734836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:18.735737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:14:18.735737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:18.735737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:18.735737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:18.735737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:18.735737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:18.735737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:18.735737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:18.735737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:18.735737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:18.735737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:18.735737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:18.735737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:18.735737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:18.735737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:18.735737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:18.735737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:18.735737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:18.735737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:18.736635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:18.736635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.736635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:18.777933 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:18.778346 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:18.778550 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:18.778790 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:14:18.779318 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:18.779700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.780650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:18.780650 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:18.780650 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:18.780650 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:18.780650 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:18.780650 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:18.780650 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.780650 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:18.782107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.782107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:14:18.782107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:18.782107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:18.782107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:18.782107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:18.782107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:18.782107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:18.782107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:18.782107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:18.782107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:18.782107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:14:18.782107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:18.782107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:14:18.782107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:14:18.782815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:18.782815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:14:18.782815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:18.782815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:14:18.782815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:14:18.782815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:18.782815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:14:18.782815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.782815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:14:18.783547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.783547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:18.783547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:18.783547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:18.783547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:18.783547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:14:18.783547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:14:18.783547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:14:18.783547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:18.783547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:18.783547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:18.783547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:18.784154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:18.784154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:18.784154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:18.784154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:18.784154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:18.784154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:18.784154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:18.784154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:18.784154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:18.784154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:18.784154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:18.784154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:18.784154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:18.784154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:18.784816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:18.784816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:18.784816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.784816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:18.785405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.785405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:18.785405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:18.785405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:18.785405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:18.785405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:18.785405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:18.785405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:18.785405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:18.785405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:18.785405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:18.785405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:18.785405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:18.785405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:18.785405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:18.785405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:18.786049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:14:18.786049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:14:18.786049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:14:18.786049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:18.786049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:18.786049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:18.786049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:18.786049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:18.786049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:18.786049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:18.786049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:18.786049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:18.786049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:18.786049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:18.786049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:18.786049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:18.786049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:18.786049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:18.786567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:18.786567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:18.786567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:18.786567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:18.786567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:18.786567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.786567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:18.786567 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:18.786567 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:18.786867 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:18.786985 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:18.787528 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:18.788002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.788789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:18.788789 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:18.788789 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:18.788789 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:18.788789 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:18.788789 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:18.788789 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.788789 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:18.789692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.789692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:14:18.789692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:18.789692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:18.789692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:18.789692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:18.789692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:18.789692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:18.789692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:18.789692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:18.789692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:18.789692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:14:18.789692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:18.789692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:14:18.789692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:14:18.790091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:18.790091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:14:18.790091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:18.790091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:14:18.790091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:14:18.790091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:18.790091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:14:18.790091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.790091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:14:18.790587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.790587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:18.790587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:18.790587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:18.790587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:18.790587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:14:18.790587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:14:18.790587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:18.790587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:18.790587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:18.790587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:18.790932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:18.790932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:18.790932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:18.790932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:18.790932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:18.790932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:18.790932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:18.790932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:18.790932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:18.790932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:18.790932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:18.790932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:18.790932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:18.790932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:18.791311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:18.791311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:18.791311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.791311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:18.791311 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:18.792013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.792013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:18.792013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:18.792013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:18.792013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:18.792013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:18.792013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:18.792013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:18.792013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:18.792013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:18.792013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:18.792013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:18.792013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:18.792013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:18.792013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:18.792013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:18.792013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:14:18.792614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:14:18.792614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:18.792614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:18.792614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:18.792614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:18.792614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:18.792614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:18.792614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:18.792614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:18.792614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:18.792614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:18.792614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:18.792614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:18.792614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:18.792614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:18.792614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:18.792614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:18.792614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:18.792614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:18.793125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:18.793125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:18.793125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.793125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:18.793125 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:18.793125 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:18.793125 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:18.793125 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:18.793374 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:18.793559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.794288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:18.794288 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:18.794288 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:18.794288 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:18.794288 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:18.794288 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:18.794288 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.794288 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:18.795126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.795126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:18.795126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:18.795126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:18.795126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:18.795126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:14:18.795126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:18.795126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:18.795126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:18.795126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:18.795126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:18.795126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:18.795464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:18.795464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:18.795464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:18.795464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:18.795464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:18.795464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:18.795464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:18.795464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:18.795464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:18.795464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:18.795464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:18.795464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:18.795464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:18.795832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.795832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:18.796112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.796112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:18.796112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:18.796112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:18.796112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:18.796112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:18.796112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:18.796112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:18.796112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:18.796112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:18.796112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:18.796112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:18.796112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:18.796112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:18.796112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:18.796112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:18.796636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:14:18.796636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:18.796636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:18.796636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:18.796636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:18.796636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:18.796636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:18.796636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:18.796636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:18.796636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:18.796636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:18.796636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:18.796636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:18.796636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:18.796636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:18.796636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:18.796636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:18.796636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:18.796636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:18.797166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:18.797166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:18.797166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:22.951973 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:22.952844 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:22.952844 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:22.953908 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:22.986964 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:22.987332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:22.988279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:22.988279 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:22.988279 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:22.988279 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:22.988279 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:22.988279 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:22.988279 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:22.988279 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:22.989343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:22.989343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:22.989343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:22.989343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:22.989343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:22.989343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:14:22.989343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:22.989343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:22.989343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:22.989343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:22.989343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:22.989343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:22.989966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:22.989966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:22.989966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:22.989966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:22.989966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:22.989966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:22.989966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:22.989966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:22.989966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:22.989966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:22.989966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:22.989966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:22.989966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:22.990705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:22.990705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:22.990705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:22.990705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:22.990705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:22.990705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:22.990705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:22.990705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:22.990705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:22.990705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:22.990705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:22.990705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:22.990705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:22.990705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:22.990705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:22.991484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:22.991484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:22.991484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:22.991484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:14:22.991484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:22.991484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:22.991484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:22.991484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:22.991484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:22.991484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:22.991484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:22.991484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:22.991484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:22.991484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:22.991484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:22.991484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:22.991484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:22.991484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:22.991484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:22.992262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:22.992262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:22.992262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:22.992262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:22.992262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:22.992262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:23.728969 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:23.729412 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:23.729643 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:23.730829 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:23.731153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.732075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:23.732075 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:23.732075 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:23.732075 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:23.732075 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:23.732075 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:23.732075 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.732075 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:23.733618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.733618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:23.733618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:23.733618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:23.733618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:23.733618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:14:23.733618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:23.733618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:23.733618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:23.733618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:23.733618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:23.733618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:23.734430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:23.734430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:23.734430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:23.734430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:23.734430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:23.734430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:23.734430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:23.734430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:23.734430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:23.734430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:23.734430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:23.734430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:23.734430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:23.734937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.734937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:23.735433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.735433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:23.735433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:23.735433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:23.735433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:23.735433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:23.735433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:23.735433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:23.735433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:23.735433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:23.735433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:23.735433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:23.735433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:23.735433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:23.735433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:23.735433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:23.736885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:14:23.736885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:23.736885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:23.736885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:23.736885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:23.736885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:23.736885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:23.736885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:23.736885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:23.736885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:23.736885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:23.736885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:23.736885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:23.736885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:23.736885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:23.736885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:23.736885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:23.736885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:23.736885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:23.737725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:23.737725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.737725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:23.779115 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:23.779470 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:23.779667 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:23.779945 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:14:23.780532 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:23.780920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.781717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:23.781717 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:23.781717 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:23.781717 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:23.781717 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:23.781717 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:23.781717 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.781717 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:23.783001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.783001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:14:23.783001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:23.783001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:23.783001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:23.783001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:23.783001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:23.783001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:23.783001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:23.783001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:23.783001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:23.783001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:14:23.783001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:23.783001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:14:23.783001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:14:23.783635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:23.783635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:14:23.783635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:23.783635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:14:23.783635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:14:23.783635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:23.783635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:14:23.783635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.783635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:14:23.784536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.784536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:23.784536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:23.784536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:23.784536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:23.784536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:14:23.784536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:14:23.784536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:14:23.784536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:23.784536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:23.784536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:23.784536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:23.785623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:23.785623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:23.785623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:23.785623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:23.785623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:23.785623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:23.785623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:23.785623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:23.785623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:23.785623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:23.785623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:23.785623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:23.785623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:23.785623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:23.786464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:23.786464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:23.786464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.786464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:23.786464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.786464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:23.786464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:23.786464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:23.786464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:23.786464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:23.786464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:23.786464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:23.786464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:23.786464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:23.786945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:23.786945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:23.786945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:23.786945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:23.786945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:23.786945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:23.786945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:14:23.786945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:14:23.786945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:14:23.786945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:23.786945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:23.786945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:23.786945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:23.786945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:23.786945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:23.786945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:23.786945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:23.786945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:23.786945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:23.787391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:23.787391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:23.787391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:23.787391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:23.787391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:23.787391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:23.787391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:23.787391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:23.787391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:23.787391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:23.787391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.787391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:23.787391 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:23.787391 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:23.787772 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:23.787772 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:23.787834 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:23.788188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.788663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:23.788663 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:23.788663 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:23.788663 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:23.788663 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:23.788663 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:23.788663 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.788663 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:23.789739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.789739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:14:23.789739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:23.789739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:23.789739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:23.789739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:23.789739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:23.789739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:23.789739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:23.789739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:23.789739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:23.789739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:14:23.789739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:23.789739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:14:23.789739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:14:23.790714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:23.790714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:14:23.790714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:23.790714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:14:23.790714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:14:23.790714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:23.790714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:14:23.790714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.790714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:14:23.791242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.791242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:23.791242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:23.791242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:23.791242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:23.791242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:14:23.791242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:14:23.791242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:23.791242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:23.791242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:23.791242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:23.791854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:23.791854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:23.791854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:23.791854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:23.791854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:23.791854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:23.791854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:23.791854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:23.791854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:23.791854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:23.791854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:23.791854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:23.791854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:23.791854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:23.792181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:23.792181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:23.792181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.792181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:23.792181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.792181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:23.792181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:23.792181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:23.792181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:23.792181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:23.792181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:23.792181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:23.792181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:23.792181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:23.792604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:23.792604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:23.792604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:23.792604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:23.792604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:23.792604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:23.792604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:14:23.792604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:14:23.792604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:23.792604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:23.792604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:23.792604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:23.792604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:23.792604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:23.792604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:23.792604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:23.792604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:23.792604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:23.792604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:23.793149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:23.793149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:23.793149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:23.793149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:23.793149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:23.793149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:23.793149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:23.793149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:23.793149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:23.793149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.793149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:23.793149 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:23.793149 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:23.793497 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:23.793497 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:23.793653 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:23.794233 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:23.794500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.795450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:23.795450 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:23.795450 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:23.795450 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:23.795450 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:23.795450 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:23.795450 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.795450 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:23.796293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.796293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:23.796293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:23.796293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:23.796293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:23.796293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:14:23.796293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:23.796293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:23.796293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:23.796293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:23.796293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:23.796293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:23.796881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:23.796881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:23.796881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:23.796881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:23.796881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:23.796881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:23.796881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:23.796881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:23.796881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:23.796881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:23.796881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:23.796881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:23.796881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:23.797415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.797415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:23.797415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.797415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:23.797415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:23.797415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:23.797415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:23.797415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:23.797415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:23.797415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:23.797415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:23.797415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:23.797415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:23.797415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:23.797415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:23.797887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:23.797887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:23.797887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:23.797887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:14:23.797887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:23.797887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:23.797887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:23.797887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:23.797887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:23.797887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:23.797887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:23.797887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:23.797887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:23.797887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:23.797887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:23.797887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:23.797887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:23.797887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:23.797887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:23.798221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:23.798221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:23.798221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:23.798221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:23.798221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:23.798221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:27.952586 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:27.953666 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:27.953666 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:27.954955 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:27.986127 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:27.986648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:27.987459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:27.987459 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:27.987459 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:27.987459 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:27.987459 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:27.987459 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:27.987459 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:27.987459 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:27.988160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:27.988160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:27.988160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:27.988160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:27.988160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:27.988160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:14:27.988160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:27.988160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:27.988160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:27.988160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:27.988160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:27.988160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:27.988803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:27.988803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:27.988803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:27.988803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:27.988803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:27.988803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:27.988803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:27.988803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:27.988803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:27.988803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:27.988803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:27.988803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:27.988803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:27.989439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:27.989439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:27.989439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:27.989439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:27.989439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:27.989439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:27.989439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:27.989439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:27.989439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:27.989439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:27.989439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:27.989439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:27.989439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:27.989439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:27.989439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:27.990216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:27.990216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:27.990216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:27.990216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:14:27.990216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:27.990216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:27.990216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:27.990216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:27.990216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:27.990216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:27.990216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:27.990216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:27.990216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:27.990216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:27.990216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:27.990216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:27.990216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:27.990216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:27.990216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:27.991200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:27.991200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:27.991200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:27.991200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:27.991200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:27.991200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:28.729831 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:28.730702 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:28.730943 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:28.732130 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:28.732573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.733751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:28.733751 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:28.733751 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:28.733751 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:28.733751 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:28.733751 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:28.733751 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.733751 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:28.735622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.735622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:28.735622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:28.735622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:28.735622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:28.735622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:14:28.735622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:28.735622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:28.735622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:28.735622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:28.735622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:28.735622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:28.736318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:28.736318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:28.736318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:28.736318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:28.736318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:28.736318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:28.736318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:28.736318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:28.736318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:28.736318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:28.736318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:28.736318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:28.736318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:28.736923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.736923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:28.736923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.736923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:28.736923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:28.736923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:28.736923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:28.736923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:28.736923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:28.736923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:28.736923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:28.736923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:28.736923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:28.736923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:28.736923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:28.737615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:28.737615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:28.737615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:28.737615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:14:28.737615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:28.737615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:28.737615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:28.737615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:28.737615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:28.737615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:28.737615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:28.737615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:28.737615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:28.737615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:28.737615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:28.737615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:28.737615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:28.737615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:28.737615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:28.738475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:28.738475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:28.738475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:28.738475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:28.738475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.738475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:28.779634 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:28.780258 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:28.780392 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:28.780509 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:14:28.781149 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:28.781471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.782337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:28.782337 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:28.782337 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:28.782337 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:28.782337 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:28.782337 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:28.782337 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.782337 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:28.783322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.783322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:14:28.783322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:28.783322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:28.783322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:28.783322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:28.783322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:28.783322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:28.783322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:28.783322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:28.783322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:28.783322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:14:28.783322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:28.783322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:14:28.783322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:14:28.784219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:28.784219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:14:28.784219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:28.784219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:14:28.784219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:14:28.784219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:28.784219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:14:28.784219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.784219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:14:28.784219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.784219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:28.784219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:28.784219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:28.784219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:28.785079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:14:28.785079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:14:28.785079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:14:28.785079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:28.785079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:28.785079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:28.785079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:28.785079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:28.785079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:28.785079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:28.785079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:28.785079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:28.785079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:28.785768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:28.785768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:28.785768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:28.785768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:28.785768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:28.785768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:28.785768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:28.785768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:28.785768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:28.785768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:28.785768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.785768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:28.786402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.786402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:28.786402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:28.786402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:28.786402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:28.786402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:28.786402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:28.786402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:28.786402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:28.786402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:28.786402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:28.786402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:28.786402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:28.786402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:28.786402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:28.786402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:28.787203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:14:28.787203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:14:28.787203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:14:28.787203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:28.787203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:28.787203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:28.787203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:28.787203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:28.787203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:28.787203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:28.787203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:28.787203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:28.787203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:28.787203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:28.787203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:28.787203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:28.787203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:28.787203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:28.788020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:28.788020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:28.788020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:28.788020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:28.788020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:28.788020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.788020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:28.788020 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:28.788020 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:28.788020 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:28.788334 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:28.788427 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:28.788834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.789607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:28.789607 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:28.789607 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:28.789607 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:28.789607 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:28.789607 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:28.789607 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.789607 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:28.790413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.790413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:14:28.790413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:28.790413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:28.790413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:28.790413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:28.790413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:28.790413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:28.790413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:28.790413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:28.790413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:28.790413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:14:28.790413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:28.790413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:14:28.790413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:14:28.790891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:28.790891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:14:28.790891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:28.790891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:14:28.790891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:14:28.790891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:28.790891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:14:28.790891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.790891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:14:28.791327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.791327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:28.791327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:28.791327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:28.791327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:28.791327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:14:28.791327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:14:28.791327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:28.791327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:28.791327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:28.791327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:28.791714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:28.791714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:28.791714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:28.791714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:28.791714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:28.791714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:28.791714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:28.791714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:28.791714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:28.791714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:28.791714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:28.791714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:28.791714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:28.791714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:28.792105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:28.792105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:28.792105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.792105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:28.792373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.792373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:28.792373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:28.792373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:28.792373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:28.792373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:28.792373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:28.792373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:28.792373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:28.792373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:28.792373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:28.792373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:28.792373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:28.792373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:28.792373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:28.792373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:28.792373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:14:28.792889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:14:28.792889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:28.792889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:28.792889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:28.792889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:28.792889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:28.792889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:28.792889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:28.792889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:28.792889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:28.792889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:28.792889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:28.792889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:28.792889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:28.792889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:28.792889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:28.792889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:28.792889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:28.792889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:28.793416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:28.793416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:28.793416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.793416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:28.793416 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:28.793416 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:28.793416 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:28.793416 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:28.793797 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:28.794143 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:28.794335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.795160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:28.795160 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:28.795160 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:28.795160 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:28.795160 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:28.795160 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:28.795160 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.795160 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:28.796057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.796057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:28.796057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:28.796057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:28.796057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:28.796057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:14:28.796057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:28.796057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:28.796057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:28.796057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:28.796057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:28.796057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:28.796426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:28.796426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:28.796426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:28.796426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:28.796426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:28.796426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:28.796426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:28.796426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:28.796426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:28.796426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:28.796426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:28.796426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:28.796426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:28.796941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.796941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:28.797010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.797010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:28.797010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:28.797010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:28.797010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:28.797010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:28.797010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:28.797010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:28.797010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:28.797010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:28.797010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:28.797010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:28.797010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:28.797010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:28.797010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:28.797010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:28.797586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:14:28.797586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:28.797586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:28.797586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:28.797586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:28.797586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:28.797586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:28.797586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:28.797586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:28.797586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:28.797586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:28.797586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:28.797586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:28.797586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:28.797586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:28.797586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:28.797586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:28.797586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:28.797586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:28.798162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:28.798162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:28.798162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:32.953336 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:32.954236 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:32.954236 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:32.955662 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:32.985629 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:32.986106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:32.987159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:32.987159 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:32.987159 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:32.987159 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:32.987159 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:32.987159 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:32.987159 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:32.987159 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:32.988032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:32.988032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:32.988032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:32.988032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:32.988032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:32.988032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:14:32.988032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:32.988032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:32.988032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:32.988032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:32.988032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:32.988032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:32.988566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:32.988566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:32.988566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:32.988566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:32.988566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:32.988566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:32.988566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:32.988566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:32.988566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:32.988566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:32.988566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:32.988566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:32.988566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:32.989118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:32.989118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:32.989118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:32.989118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:32.989118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:32.989118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:32.989118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:32.989118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:32.989118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:32.989118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:32.989118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:32.989118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:32.989118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:32.989118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:32.989118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:32.990441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:32.990441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:32.990441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:32.990441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:14:32.990441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:32.990441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:32.990441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:32.990441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:32.990441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:32.990441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:32.990441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:32.990441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:32.990441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:32.990441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:32.990441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:32.990441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:32.990441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:32.990441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:32.990441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:32.991537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:32.991537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:32.991537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:32.991537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:32.991537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:32.991537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:33.730613 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:33.731463 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:33.731463 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:33.732507 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:33.733854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.734503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:33.734503 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:33.734503 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:33.734503 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:33.734503 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:33.734503 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:33.734503 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.734503 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:33.737034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.737034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:33.737034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:33.737034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:33.737034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:33.737034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:14:33.737034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:33.737034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:33.737034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:33.737034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:33.737034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:33.737034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:33.737393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:33.737393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:33.737393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:33.737393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:33.737393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:33.737393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:33.737393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:33.737393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:33.737393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:33.737393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:33.737393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:33.737393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:33.737393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:33.737732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.737732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:33.737732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.737732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:33.737732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:33.737732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:33.737732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:33.737732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:33.737732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:33.737732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:33.737732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:33.737732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:33.737732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:33.737732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:33.737732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:33.738117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:33.738117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:33.738117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:33.738117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:14:33.738117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:33.738117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:33.738117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:33.738117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:33.738117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:33.738117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:33.738117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:33.738117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:33.738117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:33.738117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:33.738117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:33.738117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:33.738117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:33.738117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:33.738117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:33.738594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:33.738594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:33.738594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:33.738594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:33.738594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.738594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:33.780154 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:33.781077 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:33.781077 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:33.781077 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:14:33.781832 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:33.782130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.783035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:33.783035 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:33.783035 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:33.783035 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:33.783035 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:33.783035 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:33.783035 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.783035 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:33.784441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.784441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:14:33.784441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:33.784441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:33.784441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:33.784441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:33.784441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:33.784441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:33.784441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:33.784441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:33.784441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:33.784441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:14:33.784441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:33.784441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:14:33.784441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:14:33.785239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:33.785239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:14:33.785239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:33.785239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:14:33.785239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:14:33.785239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:33.785239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:14:33.785239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.785239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:14:33.786246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.786246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:33.786246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:33.786246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:33.786246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:33.786246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:14:33.786246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:14:33.786246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:14:33.786246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:33.786246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:33.786246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:33.786246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:33.787399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:33.787399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:33.787399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:33.787399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:33.787399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:33.787399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:33.787399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:33.787399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:33.787399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:33.787399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:33.787399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:33.787399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:33.787399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:33.787399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:33.788646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:33.788646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:33.788646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.788646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:33.788646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.788646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:33.788646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:33.788646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:33.788646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:33.788646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:33.788646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:33.788646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:33.788646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:33.788646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:33.789106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:33.789106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:33.789106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:33.789106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:33.789106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:33.789106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:33.789106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:14:33.789106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:14:33.789106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:14:33.789106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:33.789106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:33.789106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:33.789106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:33.789106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:33.789106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:33.789106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:33.789106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:33.789106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:33.789106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:33.790660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:33.790660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:33.790660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:33.790660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:33.790660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:33.790660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:33.790660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:33.790660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:33.790660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:33.790660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:33.790660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.790660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:33.790660 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:33.790660 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:33.791149 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:33.791149 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:33.791149 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:33.791149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.791328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:33.791328 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:33.791328 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:33.791328 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:33.791328 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:33.791328 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:33.791328 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.791328 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:33.792237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.792237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:14:33.792237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:33.792237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:33.792237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:33.792237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:33.792237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:33.792237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:33.792237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:33.792237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:33.792237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:33.792237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:14:33.792237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:33.792237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:14:33.792237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:14:33.793575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:33.793575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:14:33.793575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:33.793575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:14:33.793575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:14:33.793575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:33.793575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:14:33.793575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.793575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:14:33.793575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.793575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:33.793575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:33.793575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:33.793575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:33.794034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:14:33.794034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:14:33.794034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:33.794034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:33.794034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:33.794034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:33.794034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:33.794034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:33.794034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:33.794034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:33.794034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:33.794034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:33.794034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:33.794345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:33.794345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:33.794345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:33.794345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:33.794345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:33.794345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:33.794345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:33.794345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:33.794345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:33.794345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.794345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:33.795220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.795220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:33.795220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:33.795220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:33.795220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:33.795220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:33.795220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:33.795220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:33.795220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:33.795220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:33.795220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:33.795220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:33.795220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:33.795220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:33.795220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:33.795220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:33.795220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:14:33.795599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:14:33.795599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:33.795599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:33.795599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:33.795599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:33.795599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:33.795599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:33.795599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:33.795599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:33.795599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:33.795599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:33.795599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:33.795599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:33.795599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:33.795599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:33.795599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:33.795599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:33.795599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:33.795599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:33.796104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:33.796104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:33.796104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.796104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:33.796104 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:33.796104 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:33.796104 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:33.796104 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:33.796495 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:33.796871 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:33.797077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.797726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:33.797726 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:33.797726 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:33.797726 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:33.797726 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:33.797726 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:33.797726 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.797726 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:33.798429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.798429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:33.798429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:33.798429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:33.798429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:33.798429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:14:33.798429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:33.798429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:33.798429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:33.798429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:33.798429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:33.798429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:33.798805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:33.798805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:33.798805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:33.798805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:33.798805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:33.798805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:33.798805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:33.798805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:33.798805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:33.798805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:33.798805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:33.798805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:33.798805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:33.799207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.799207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:33.799415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.799415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:33.799415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:33.799415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:33.799415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:33.799415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:33.799415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:33.799415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:33.799415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:33.799415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:33.799415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:33.799415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:33.799415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:33.799415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:33.799415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:33.799415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:33.800052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:14:33.800052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:33.800052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:33.800052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:33.800052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:33.800052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:33.800052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:33.800052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:33.800052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:33.800052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:33.800052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:33.800052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:33.800052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:33.800052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:33.800052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:33.800052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:33.800052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:33.800052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:33.800052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:33.800401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:33.800401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:33.800401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:37.954051 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:37.955016 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:37.955016 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:37.956160 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:37.992175 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:37.992580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:37.993510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:37.993510 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:37.993510 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:37.993510 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:37.993510 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:37.993510 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:37.993510 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:37.993510 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:37.994794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:37.994794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:37.994794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:37.994794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:37.994794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:37.994794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:14:37.994794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:37.994794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:37.994794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:37.994794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:37.994794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:37.994794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:37.995372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:37.995372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:37.995372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:37.995372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:37.995372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:37.995372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:37.995372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:37.995372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:37.995372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:37.995372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:37.995372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:37.995372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:37.995372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:37.995950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:37.995950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:37.996275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:37.996275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:37.996275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:37.996275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:37.996275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:37.996275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:37.996275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:37.996275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:37.996275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:37.996275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:37.996275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:37.996275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:37.996275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:37.996275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:37.996275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:37.996275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:37.997130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:14:37.997130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:37.997130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:37.997130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:37.997130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:37.997130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:37.997130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:37.997130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:37.997130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:37.997130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:37.997130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:37.997130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:37.997130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:37.997130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:37.997130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:37.997130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:37.997130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:37.997130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:37.997130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:37.997981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:37.997981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:37.997981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:38.731217 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:38.731860 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:38.732050 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:38.733305 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:38.733764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.734868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:38.734868 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:38.734868 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:38.734868 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:38.734868 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:38.734868 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:38.734868 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.734868 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:38.736395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.736395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:38.736395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:38.736395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:38.736395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:38.736395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:14:38.736395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:38.736395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:38.736395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:38.736395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:38.736395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:38.736395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:38.737457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:38.737457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:38.737457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:38.737457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:38.737457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:38.737457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:38.737457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:38.737457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:38.737457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:38.737457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:38.737457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:38.737457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:38.737457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:38.738190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.738190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:38.738190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.738190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:38.738190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:38.738190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:38.738190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:38.738190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:38.738190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:38.738190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:38.738190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:38.738190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:38.738190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:38.738190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:38.738190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:38.739108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:38.739108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:38.739108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:38.739108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:14:38.739108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:38.739108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:38.739108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:38.739108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:38.739108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:38.739108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:38.739108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:38.739108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:38.739108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:38.739108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:38.739108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:38.739108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:38.739108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:38.739108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:38.739108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:38.740381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:38.740381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:38.740381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:38.740381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:38.740381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.740381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:38.781308 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:38.782505 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:38.782785 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:38.783047 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:14:38.783809 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:38.784195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.785137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:38.785137 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:38.785137 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:38.785137 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:38.785137 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:38.785137 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:38.785137 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.785137 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:38.786440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.786440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:14:38.786440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:38.786440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:38.786440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:38.786440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:38.786440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:38.786440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:38.786440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:38.786440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:38.786440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:38.786440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:14:38.786440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:38.786440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:14:38.786440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:14:38.787187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:38.787187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:14:38.787187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:38.787187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:14:38.787187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:14:38.787187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:38.787187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:14:38.787187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.787187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:14:38.787771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.787771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:38.787771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:38.787771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:38.787771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:38.787771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:14:38.787771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:14:38.787771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:14:38.787771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:38.787771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:38.787771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:38.787771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:38.788524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:38.788524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:38.788524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:38.788524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:38.788524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:38.788524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:38.788524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:38.788524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:38.788524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:38.788524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:38.788524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:38.788524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:38.788524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:38.788524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:38.789253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:38.789253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:38.789253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.789253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:38.789566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.789566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:38.789566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:38.789566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:38.789566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:38.789566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:38.789566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:38.789566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:38.789566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:38.789566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:38.789566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:38.789566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:38.789566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:38.789566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:38.789566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:38.789566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:38.790322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:14:38.790322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:14:38.790322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:14:38.790322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:38.790322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:38.790322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:38.790322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:38.790322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:38.790322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:38.790322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:38.790322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:38.790322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:38.790322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:38.790322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:38.790322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:38.790322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:38.790322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:38.790322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:38.791356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:38.791356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:38.791356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:38.791356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:38.791356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:38.791356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.791356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:38.791356 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:38.791356 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:38.791356 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:38.792037 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:38.792037 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:38.792037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.792037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:38.792037 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:38.792037 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:38.792037 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:38.792037 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:38.792037 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:38.792037 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.792037 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:38.792534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.792534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:14:38.792534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:38.792534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:38.792534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:38.792534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:38.792534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:38.792534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:38.792534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:38.792534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:38.792534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:38.792534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:14:38.792534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:38.792534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:14:38.792534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:14:38.793253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:38.793253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:14:38.793253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:38.793253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:14:38.793253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:14:38.793253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:38.793253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:14:38.793253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.793253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:14:38.793703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.793703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:38.793703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:38.793703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:38.793703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:38.793703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:14:38.793703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:14:38.793703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:38.793703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:38.793703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:38.793703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:38.794213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:38.794213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:38.794213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:38.794213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:38.794213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:38.794213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:38.794213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:38.794213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:38.794213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:38.794213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:38.794213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:38.794213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:38.794213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:38.794213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:38.794881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:38.794881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:38.794881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.794881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:38.794881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.794881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:38.794881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:38.794881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:38.794881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:38.794881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:38.794881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:38.794881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:38.794881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:38.794881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:38.796012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:38.796012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:38.796012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:38.796012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:38.796012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:38.796012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:38.796012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:14:38.796012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:14:38.796012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:38.796012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:38.796012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:38.796012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:38.796012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:38.796012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:38.796012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:38.796012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:38.796012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:38.796012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:38.796012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:38.796961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:38.796961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:38.796961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:38.796961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:38.796961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:38.796961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:38.796961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:38.796961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:38.796961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:38.796961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.796961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:38.796961 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:38.796961 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:38.797701 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:38.797701 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:38.797701 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:38.797701 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:38.797701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.797701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:38.797701 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:38.797701 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:38.797701 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:38.798113 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:38.798113 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:38.798113 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.798113 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:38.798598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.798598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:38.798598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:38.798598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:38.798598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:38.798598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:14:38.798598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:38.798598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:38.798598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:38.798598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:38.798598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:38.798598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:38.798902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:38.798902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:38.798902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:38.798902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:38.798902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:38.798902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:38.798902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:38.798902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:38.798902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:38.798902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:38.798902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:38.798902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:38.798902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:38.799378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.799378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:38.799937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.799937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:38.799937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:38.799937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:38.799937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:38.799937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:38.799937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:38.799937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:38.799937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:38.799937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:38.799937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:38.799937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:38.799937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:38.799937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:38.799937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:38.799937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:38.800249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:14:38.800249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:38.800249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:38.800249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:38.800249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:38.800249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:38.800249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:38.800249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:38.800249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:38.800249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:38.800249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:38.800249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:38.800249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:38.800249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:38.800249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:38.800249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:38.800249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:38.800249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:38.800249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:38.800608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:38.800608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:38.800608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:42.954726 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:42.955806 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:42.955806 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:42.956560 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:42.995369 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:42.995614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:42.996423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:42.996423 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:42.996423 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:42.996423 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:42.996423 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:42.996423 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:42.996423 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:42.996423 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:42.997568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:42.997568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:42.997568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:42.997568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:42.997568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:42.997568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:14:42.997568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:42.997568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:42.997568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:42.997568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:42.997568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:42.997568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:42.998312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:42.998312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:42.998312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:42.998312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:42.998312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:42.998312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:42.998312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:42.998312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:42.998312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:42.998312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:42.998312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:42.998312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:42.998312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:42.998971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:42.998971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:42.998971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:42.998971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:42.998971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:42.998971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:42.998971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:42.998971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:42.998971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:42.998971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:42.998971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:42.998971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:42.998971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:42.998971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:42.998971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:42.999668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:42.999668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:42.999668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:42.999668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:14:42.999668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:42.999668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:42.999668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:42.999668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:42.999668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:42.999668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:42.999668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:42.999668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:42.999668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:42.999668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:42.999668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:42.999668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:42.999668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:42.999668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:42.999668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:43.000554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:43.000554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:43.000554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:43.000554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:43.000554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.000554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:43.732220 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:43.732650 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:43.732903 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:43.733714 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:43.734000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.734926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:43.734926 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:43.734926 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:43.734926 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:43.734926 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:43.734926 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:43.734926 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.734926 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:43.736622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.736622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:43.736622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:43.736622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:43.736622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:43.736622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:14:43.736622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:43.736622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:43.736622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:43.736622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:43.736622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:43.736622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:43.737598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:43.737598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:43.737598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:43.737598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:43.737598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:43.737598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:43.737598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:43.737598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:43.737598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:43.737598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:43.737598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:43.737598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:43.737598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:43.738385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.738385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:43.739214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.739214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:43.739214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:43.739214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:43.739214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:43.739214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:43.739214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:43.739214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:43.739214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:43.739214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:43.739214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:43.739214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:43.739214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:43.739214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:43.739214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:43.739214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:43.740892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:14:43.740892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:43.740892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:43.740892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:43.740892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:43.740892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:43.740892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:43.740892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:43.740892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:43.740892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:43.740892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:43.740892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:43.740892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:43.740892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:43.740892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:43.740892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:43.740892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:43.740892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:43.740892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:43.742003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:43.742003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.742003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:43.782403 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:43.782927 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:43.783138 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:43.783428 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:14:43.784120 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:43.784471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.785470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:43.785470 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:43.785470 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:43.785470 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:43.785470 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:43.785470 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:43.785470 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.785470 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:43.787005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.787005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:14:43.787005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:43.787005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:43.787005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:43.787005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:43.787005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:43.787005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:43.787005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:43.787005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:43.787005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:43.787005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:14:43.787005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:43.787005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:14:43.787005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:14:43.788092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:43.788092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:14:43.788092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:43.788092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:14:43.788092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:14:43.788092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:43.788092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:14:43.788092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.788092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:14:43.788909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.788909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:43.788909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:43.788909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:43.788909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:43.788909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:14:43.788909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:14:43.788909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:14:43.788909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:43.788909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:43.788909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:43.788909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:43.790629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:43.790629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:43.790629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:43.790629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:43.790629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:43.790629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:43.790629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:43.790629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:43.790629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:43.790629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:43.790629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:43.790629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:43.790629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:43.790629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:43.791084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:43.791084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:43.791084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.791084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:43.791084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.791084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:43.791084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:43.791084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:43.791084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:43.791084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:43.791084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:43.791084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:43.791084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:43.791084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:43.792465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:43.792465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:43.792465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:43.792465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:43.792465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:43.792465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:43.792465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:14:43.792465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:14:43.792465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:14:43.792465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:43.792465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:43.792465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:43.792465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:43.792465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:43.792465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:43.792465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:43.792465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:43.792465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:43.792465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:43.793032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:43.793032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:43.793032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:43.793032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:43.793032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:43.793032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:43.793032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:43.793032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:43.793032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:43.793032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:43.793032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.793032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:43.793032 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:43.793032 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:43.793379 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:43.793379 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:43.793379 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:43.793379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.793927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:43.793927 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:43.793927 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:43.793927 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:43.793927 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:43.793927 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:43.793927 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.793927 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:43.794647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.794647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:14:43.794647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:43.794647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:43.794647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:43.794647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:43.794647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:43.794647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:43.794647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:43.794647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:43.794647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:43.794647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:14:43.794647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:43.794647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:14:43.794647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:14:43.795930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:43.795930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:14:43.795930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:43.795930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:14:43.795930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:14:43.795930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:43.795930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:14:43.795930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.795930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:14:43.795930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.795930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:43.795930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:43.795930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:43.795930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:43.796586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:14:43.796586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:14:43.796586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:43.796586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:43.796586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:43.796586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:43.796586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:43.796586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:43.796586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:43.796586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:43.796586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:43.796586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:43.796586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:43.797050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:43.797050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:43.797050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:43.797050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:43.797050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:43.797050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:43.797050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:43.797050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:43.797050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:43.797050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.797050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:43.797736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.797736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:43.797736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:43.797736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:43.797736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:43.797736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:43.797736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:43.797736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:43.797736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:43.797736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:43.797736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:43.797736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:43.797736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:43.797736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:43.797736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:43.797736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:43.797736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:14:43.798378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:14:43.798378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:43.798378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:43.798378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:43.798378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:43.798378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:43.798378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:43.798378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:43.798378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:43.798378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:43.798378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:43.798378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:43.798378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:43.798378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:43.798378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:43.798378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:43.798378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:43.798378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:43.798378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:43.798897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:43.798897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:43.798897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.798897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:43.798897 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:43.798897 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:43.798897 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:43.798897 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:43.799364 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:43.799364 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:43.799364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.799646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:43.799646 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:43.799646 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:43.799646 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:43.799646 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:43.799646 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:43.799646 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.799646 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:43.800376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.800376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:43.800376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:43.800376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:43.800376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:43.800376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:14:43.800376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:43.800376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:43.800376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:43.800376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:43.800376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:43.800376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:43.800740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:43.800740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:43.800740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:43.800740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:43.800740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:43.800740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:43.800740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:43.800740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:43.800740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:43.800740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:43.800740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:43.800740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:43.800740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:43.801066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.801066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:43.801412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.801412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:43.801412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:43.801412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:43.801412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:43.801412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:43.801412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:43.801412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:43.801412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:43.801412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:43.801412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:43.801412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:43.801412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:43.801412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:43.801412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:43.801412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:43.802226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:14:43.802226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:43.802226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:43.802226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:43.802226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:43.802226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:43.802226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:43.802226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:43.802226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:43.802226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:43.802226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:43.802226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:43.802226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:43.802226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:43.802226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:43.802226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:43.802226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:43.802226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:43.802226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:43.802780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:43.802780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:43.802780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:47.955306 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:47.956758 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:47.956758 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:47.957431 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:47.984428 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:47.984610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:47.985271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:47.985271 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:47.985271 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:47.985271 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:47.985271 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:47.985271 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:47.985271 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:47.985271 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:47.986070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:47.986070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:47.986070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:47.986070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:47.986070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:47.986070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:14:47.986070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:47.986070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:47.986070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:47.986070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:47.986070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:47.986070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:47.986798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:47.986798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:47.986798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:47.986798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:47.986798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:47.986798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:47.986798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:47.986798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:47.986798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:47.986798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:47.986798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:47.986798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:47.986798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:47.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:47.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:47.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:47.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:47.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:47.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:47.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:47.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:47.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:47.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:47.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:47.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:47.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:47.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:47.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:47.988310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:47.988310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:47.988310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:47.988310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:14:47.988310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:47.988310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:47.988310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:47.988310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:47.988310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:47.988310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:47.988310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:47.988310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:47.988310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:47.988310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:47.988310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:47.988310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:47.988310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:47.988310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:47.988310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:47.989393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:47.989393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:47.989393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:47.989393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:47.989393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:47.989393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:48.732862 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:48.733425 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:48.733710 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:48.734524 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:48.734866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.735703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:48.735703 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:48.735703 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:48.735703 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:48.735703 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:48.735703 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:48.735703 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.735703 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:48.737403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.737403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:48.737403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:48.737403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:48.737403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:48.737403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:14:48.737403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:48.737403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:48.737403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:48.737403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:48.737403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:48.737403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:48.738108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:48.738108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:48.738108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:48.738108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:48.738108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:48.738108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:48.738108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:48.738108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:48.738108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:48.738108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:48.738108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:48.738108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:48.738108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:48.738665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.738665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:48.739301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.739301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:48.739301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:48.739301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:48.739301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:48.739301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:48.739301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:48.739301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:48.739301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:48.739301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:48.739301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:48.739301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:48.739301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:48.739301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:48.739301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:48.739301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:48.740147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:14:48.740147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:48.740147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:48.740147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:48.740147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:48.740147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:48.740147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:48.740147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:48.740147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:48.740147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:48.740147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:48.740147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:48.740147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:48.740147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:48.740147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:48.740147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:48.740147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:48.740147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:48.740147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:48.741038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:48.741038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.741038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:48.782946 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:48.783506 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:48.783703 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:48.783912 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:14:48.784381 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:48.784729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.785485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:48.785485 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:48.785485 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:48.785485 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:48.785485 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:48.785485 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:48.785485 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.785485 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:48.786839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.786839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:14:48.786839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:48.786839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:48.786839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:48.786839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:48.786839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:48.786839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:48.786839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:48.786839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:48.786839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:48.786839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:14:48.786839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:48.786839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:14:48.786839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:14:48.787352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:48.787352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:14:48.787352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:48.787352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:14:48.787352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:14:48.787352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:48.787352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:14:48.787352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.787352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:14:48.788170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.788170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:48.788170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:48.788170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:48.788170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:48.788170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:14:48.788170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:14:48.788170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:14:48.788170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:48.788170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:48.788170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:48.788170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:48.789007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:48.789007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:48.789007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:48.789007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:48.789007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:48.789007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:48.789007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:48.789007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:48.789007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:48.789007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:48.789007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:48.789007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:48.789007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:48.789007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:48.789924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:48.789924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:48.789924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.789924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:48.790299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.790299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:48.790299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:48.790299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:48.790299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:48.790299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:48.790299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:48.790299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:48.790299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:48.790299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:48.790299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:48.790299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:48.790299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:48.790299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:48.790299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:48.790299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:48.791132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:14:48.791132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:14:48.791132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:14:48.791132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:48.791132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:48.791132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:48.791132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:48.791132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:48.791132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:48.791132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:48.791132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:48.791132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:48.791132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:48.791132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:48.791132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:48.791132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:48.791132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:48.791132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:48.791974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:48.791974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:48.791974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:48.791974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:48.791974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:48.791974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.791974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:48.791974 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:48.792392 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:48.792392 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:48.792584 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:48.792989 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:48.793205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.793845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:48.793845 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:48.793845 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:48.793845 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:48.793845 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:48.793845 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:48.793845 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.793845 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:48.795161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.795161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:14:48.795161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:48.795161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:48.795161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:48.795161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:48.795161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:48.795161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:48.795161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:48.795161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:48.795161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:48.795161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:14:48.795161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:48.795161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:14:48.795161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:14:48.795858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:48.795858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:14:48.795858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:48.795858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:14:48.795858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:14:48.795858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:48.795858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:14:48.795858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.795858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:14:48.797032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.797032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:48.797032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:48.797032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:48.797032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:48.797032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:14:48.797032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:14:48.797032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:48.797032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:48.797032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:48.797032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:48.797453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:48.797453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:48.797453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:48.797453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:48.797453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:48.797453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:48.797453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:48.797453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:48.797453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:48.797453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:48.797453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:48.797453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:48.797453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:48.797453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:48.797871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:48.797871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:48.797871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.797871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:48.798816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.798816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:48.798816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:48.798816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:48.798816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:48.798816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:48.798816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:48.798816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:48.798816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:48.798816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:48.798816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:48.798816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:48.798816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:48.798816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:48.798816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:48.798816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:48.798816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:14:48.799374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:14:48.799374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:48.799374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:48.799374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:48.799374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:48.799374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:48.799374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:48.799374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:48.799374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:48.799374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:48.799374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:48.799374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:48.799374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:48.799374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:48.799374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:48.799374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:48.799374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:48.799374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:48.799374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:48.799903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:48.799903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:48.799903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.799903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:48.799903 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:48.799903 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:48.799903 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:48.799903 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:48.800172 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:48.800172 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:48.800172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.800623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:48.800623 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:48.800623 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:48.800623 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:48.800623 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:48.800623 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:48.800623 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.800623 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:48.801447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.801447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:48.801447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:48.801447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:48.801447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:48.801447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:14:48.801447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:48.801447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:48.801447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:48.801447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:48.801447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:48.801447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:48.801874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:48.801874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:48.801874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:48.801874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:48.801874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:48.801874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:48.801874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:48.801874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:48.801874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:48.801874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:48.801874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:48.801874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:48.801874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:48.802259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.802259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:48.802443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.802443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:48.802443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:48.802443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:48.802443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:48.802443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:48.802443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:48.802443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:48.802443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:48.802443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:48.802443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:48.802443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:48.802443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:48.802443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:48.802443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:48.802443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:48.802941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:14:48.802941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:48.802941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:48.802941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:48.802941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:48.802941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:48.802941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:48.802941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:48.802941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:48.802941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:48.802941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:48.802941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:48.802941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:48.802941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:48.802941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:48.802941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:48.802941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:48.802941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:48.802941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:48.803467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:48.803467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:48.803467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:52.956038 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:52.956790 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:52.956790 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:52.957899 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:52.991964 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:52.992342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:52.993201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:52.993201 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:52.993201 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:52.993201 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:52.993201 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:52.993201 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:52.993201 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:52.993201 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:52.993954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:52.993954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:52.993954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:52.993954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:52.993954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:52.993954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:14:52.993954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:52.993954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:52.993954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:52.993954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:52.993954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:52.993954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:52.994515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:52.994515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:52.994515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:52.994515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:52.994515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:52.994515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:52.994515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:52.994515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:52.994515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:52.994515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:52.994515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:52.994515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:52.994515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:52.995062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:52.995062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:52.995062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:52.995062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:52.995062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:52.995062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:52.995062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:52.995062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:52.995062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:52.995062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:52.995062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:52.995062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:52.995062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:52.995062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:52.995062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:52.995713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:52.995713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:52.995713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:52.995713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:14:52.995713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:52.995713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:52.995713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:52.995713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:52.995713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:52.995713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:52.995713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:52.995713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:52.995713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:52.995713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:52.995713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:52.995713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:52.995713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:52.995713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:52.995713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:52.996538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:52.996538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:52.996538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:52.996538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:52.996538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:52.996538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:53.733514 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:53.734055 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:53.734203 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:53.734930 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:53.735242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.736028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:53.736028 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:53.736028 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:53.736028 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:53.736028 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:53.736028 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:53.736028 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.736028 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:53.737150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.737150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:53.737150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:53.737150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:53.737150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:53.737150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:14:53.737150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:53.737150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:53.737150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:53.737150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:53.737150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:53.737150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:53.737772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:53.737772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:53.737772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:53.737772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:53.737772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:53.737772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:53.737772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:53.737772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:53.737772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:53.737772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:53.737772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:53.737772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:53.737772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:53.738262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.738262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:53.738778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.738778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:53.738778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:53.738778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:53.738778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:53.738778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:53.738778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:53.738778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:53.738778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:53.738778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:53.738778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:53.738778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:53.738778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:53.738778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:53.738778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:53.738778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:53.740349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:14:53.740349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:53.740349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:53.740349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:53.740349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:53.740349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:53.740349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:53.740349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:53.740349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:53.740349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:53.740349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:53.740349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:53.740349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:53.740349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:53.740349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:53.740349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:53.740349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:53.740349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:53.740349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:53.741808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:53.741808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.741808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:53.783510 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:53.784642 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:53.784642 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:53.784642 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:14:53.785069 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:53.785069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.785558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:53.785558 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:53.785558 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:53.785558 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:53.785558 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:53.785558 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:53.785558 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.785558 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:53.786806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.786806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:14:53.786806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:53.786806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:53.786806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:53.786806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:53.786806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:53.786806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:53.786806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:53.786806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:53.786806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:53.786806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:14:53.786806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:53.786806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:14:53.786806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:14:53.788225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:53.788225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:14:53.788225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:53.788225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:14:53.788225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:14:53.788225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:53.788225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:14:53.788225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.788225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:14:53.788225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.788225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:53.788225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:53.788225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:53.788225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:53.789537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:14:53.789537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:14:53.789537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:14:53.789537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:53.789537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:53.789537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:53.789537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:53.789537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:53.789537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:53.789537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:53.789537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:53.789537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:53.789537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:53.790110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:53.790110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:53.790110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:53.790110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:53.790110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:53.790110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:53.790110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:53.790110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:53.790110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:53.790110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:53.790110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.790110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:53.790581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.790581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:53.790581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:53.790581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:53.790581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:53.790581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:53.790581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:53.790581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:53.790581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:53.790581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:53.790581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:53.790581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:53.790581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:53.790581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:53.790581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:53.790581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:53.792407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:14:53.792407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:14:53.792407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:14:53.792407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:53.792407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:53.792407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:53.792407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:53.792407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:53.792407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:53.792407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:53.792407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:53.792407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:53.792407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:53.792407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:53.792407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:53.792407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:53.792407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:53.792407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:53.793739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:53.793739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:53.793739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:53.793739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:53.793739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:53.793739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.793739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:53.793739 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:53.793739 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:53.793739 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:53.793936 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:53.793936 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:53.793936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.793936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:53.793936 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:53.793936 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:53.793936 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:53.793936 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:53.793936 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:53.793936 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.793936 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:53.794341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.794341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:14:53.794341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:53.794341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:53.794341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:53.794341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:53.794341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:53.794341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:53.794341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:53.794341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:53.794341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:53.794341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:14:53.794341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:53.794341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:14:53.794341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:14:53.795013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:53.795013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:14:53.795013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:53.795013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:14:53.795013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:14:53.795013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:53.795013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:14:53.795013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.795013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:14:53.795404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.795404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:53.795404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:53.795404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:53.795404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:53.795404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:14:53.795404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:14:53.795404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:53.795404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:53.795404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:53.795404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:53.795724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:53.795724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:53.795724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:53.795724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:53.795724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:53.795724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:53.795724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:53.795724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:53.795724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:53.795724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:53.795724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:53.795724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:53.795724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:53.795724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:53.795959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:53.795959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:53.795959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.795959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:53.796197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.796197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:53.796197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:53.796197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:53.796197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:53.796197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:53.796197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:53.796197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:53.796197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:53.796197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:53.796197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:53.796197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:53.796197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:53.796197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:53.796197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:53.796197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:53.796197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:14:53.796660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:14:53.796660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:53.796660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:53.796660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:53.796660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:53.796660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:53.796660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:53.796660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:53.796660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:53.796660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:53.796660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:53.796660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:53.796660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:53.796660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:53.796660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:53.796660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:53.796660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:53.796660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:53.796660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:53.796997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:53.796997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:53.796997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.796997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:53.799336 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:53.799592 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:53.799731 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:53.799885 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:53.800239 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:53.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:53.800791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.801467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:53.801467 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:53.801467 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:53.801467 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:53.801467 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:53.801467 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:53.801467 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.801467 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:53.802317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.802317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:53.802317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:53.802317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:53.802317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:53.802317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:14:53.802317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:53.802317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:53.802317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:53.802317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:53.802317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:53.802317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:53.802817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:53.802817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:53.802817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:53.802817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:53.802817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:53.802817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:53.802817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:53.802817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:53.802817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:53.802817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:53.802817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:53.802817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:53.802817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:53.803139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.803139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:53.803768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.803768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:53.803768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:53.803768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:53.803768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:53.803768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:53.803768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:53.803768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:53.803768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:53.803768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:53.803768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:53.803768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:53.803768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:53.803768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:53.803768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:53.803768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:53.804918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:14:53.804918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:53.804918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:53.804918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:53.804918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:53.804918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:53.804918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:53.804918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:53.804918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:53.804918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:53.804918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:53.804918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:53.804918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:53.804918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:53.804918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:53.804918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:53.804918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:53.804918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:53.804918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:53.805540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:53.805540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:53.805540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:57.957045 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:57.957835 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:57.957835 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:57.959015 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:57.992660 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:57.992660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:57.992859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:57.992859 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:57.992859 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:57.992859 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:57.992859 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:57.992859 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:57.992859 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:57.992859 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:57.993698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:57.993698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:57.993698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:57.993698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:57.993698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:57.993698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:14:57.993698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:57.993698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:57.993698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:57.993698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:57.993698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:57.993698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:57.994258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:57.994258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:57.994258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:57.994258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:57.994258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:57.994258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:57.994258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:57.994258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:57.994258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:57.994258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:57.994258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:57.994258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:57.994258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:57.994815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:57.994815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:57.994815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:57.994815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:57.994815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:57.994815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:57.994815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:57.994815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:57.994815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:57.994815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:57.994815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:57.994815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:57.994815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:57.994815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:57.994815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:57.995401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:57.995401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:57.995401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:57.995401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:14:57.995401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:57.995401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:57.995401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:57.995401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:57.995401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:57.995401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:57.995401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:57.995401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:57.995401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:57.995401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:57.995401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:57.995401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:57.995401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:57.995401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:57.995401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:57.996179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:57.996179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:57.996179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:57.996179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:57.996179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:57.996179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:58.734188 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:58.734803 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:58.735005 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:58.735840 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:58.736234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.737198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:58.737198 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:58.737198 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:58.737198 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:58.737198 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:58.737198 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:58.737198 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.737198 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:58.738922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.738922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:58.738922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:58.738922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:58.738922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:58.738922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:14:58.738922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:58.738922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:58.738922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:58.738922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:58.738922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:58.738922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:58.739577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:58.739577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:58.739577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:58.739577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:58.739577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:58.739577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:58.739577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:58.739577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:58.739577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:58.739577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:58.739577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:58.739577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:58.739577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:58.740289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.740289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:58.741057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.741057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:58.741057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:58.741057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:58.741057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:58.741057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:58.741057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:58.741057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:58.741057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:58.741057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:58.741057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:58.741057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:58.741057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:58.741057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:58.741057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:58.741057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:58.742001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:14:58.742001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:58.742001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:58.742001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:58.742001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:58.742001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:58.742001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:58.742001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:58.742001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:58.742001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:58.742001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:58.742001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:58.742001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:58.742001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:58.742001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:58.742001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:58.742001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:58.742001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:58.742001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:58.742840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:58.742840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.742840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:58.784158 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:58.784578 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:58.784763 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:58.784994 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:14:58.785567 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:58.785922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.786672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:58.786672 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:58.786672 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:58.786672 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:58.786672 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:58.786672 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:58.786672 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.786672 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:58.787887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.787887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:14:58.787887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:58.787887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:58.787887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:58.787887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:58.787887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:58.787887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:58.787887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:58.787887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:58.787887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:58.787887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:14:58.787887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:58.787887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:14:58.787887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:14:58.788500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:58.788500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:14:58.788500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:58.788500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:14:58.788500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:14:58.788500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:58.788500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:14:58.788500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.788500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:14:58.789190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.789190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:58.789190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:58.789190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:58.789190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:58.789190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:14:58.789190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:14:58.789190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:14:58.789190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:58.789190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:58.789190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:58.789190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:58.789629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:58.789629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:58.789629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:58.789629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:58.789629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:58.789629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:58.789629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:58.789629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:58.789629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:58.789629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:58.789629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:58.789629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:58.789629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:58.789629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:58.790241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:58.790241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:58.790241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.790241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:58.790836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.790836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:58.790836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:58.790836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:58.790836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:58.790836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:58.790836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:58.790836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:58.790836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:58.790836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:58.790836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:58.790836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:58.790836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:58.790836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:58.790836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:58.790836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:58.791290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:14:58.791290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:14:58.791290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:14:58.791290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:58.791290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:58.791290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:58.791290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:58.791290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:58.791290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:58.791290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:58.791290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:58.791290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:58.791290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:58.791290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:58.791290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:58.791290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:58.791290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:58.791290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:58.792004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:58.792004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:58.792004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:58.792004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:58.792004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:58.792004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.792004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:58.792432 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:58.792831 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:58.792948 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:58.793072 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:58.793478 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:58.793814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.794320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:58.794320 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:58.794320 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:58.794320 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:58.794320 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:58.794320 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:58.794320 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.794320 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:58.795068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.795068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:14:58.795068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:58.795068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:58.795068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:58.795068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:58.795068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:58.795068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:58.795068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:58.795068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:14:58.795068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:58.795068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:14:58.795068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:58.795068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:14:58.795068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:14:58.796608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:58.796608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:14:58.796608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:58.796608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:14:58.796608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:14:58.796608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:58.796608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:14:58.796608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.796608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:14:58.796608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.796608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:58.796608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:58.796608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:58.796608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:58.797127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:14:58.797127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:14:58.797127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:58.797127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:58.797127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:58.797127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:58.797127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:58.797127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:58.797127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:58.797127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:58.797127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:58.797127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:58.797127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:58.797551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:58.797551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:58.797551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:58.797551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:58.797551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:58.797551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:58.797551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:58.797551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:58.797551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:58.797551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.797551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:58.797551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.798329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:58.798329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:58.798329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:58.798329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:58.798329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:58.798329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:58.798329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:58.798329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:58.798329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:58.798329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:58.798329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:58.798329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:58.798329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:58.798329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:58.798329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:58.798329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:14:58.798329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:14:58.798329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:14:58.798329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:14:58.799244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:58.799244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:58.799244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:58.799244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:58.799244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:58.799244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:58.799244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:58.799244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:58.799244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:58.799244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:58.799244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:58.799244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:58.799244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:58.799244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:58.799244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:58.799244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:58.799244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:58.799244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:58.799244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.799244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:14:58.800140 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:14:58.800140 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:14:58.800140 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:14:58.800310 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:14:58.800600 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:58.800846 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:14:58.800954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.801459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:14:58.801459 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:58.801459 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:14:58.801459 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:14:58.801459 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:58.801459 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:14:58.801459 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.801459 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:14:58.802270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.802270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:14:58.802270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:58.802270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:14:58.802270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:58.802270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:14:58.802270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:58.802270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:58.802270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:58.802270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:58.802270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:58.802270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:58.802672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:14:58.802672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:58.802672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:14:58.802672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:58.802672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:14:58.802672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:14:58.802672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:58.802672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:14:58.802672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:58.802672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:14:58.802672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:14:58.802672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:58.802672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:14:58.804217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.804217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:14:58.804217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.804217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:14:58.804217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:14:58.804217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:14:58.804217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:14:58.804217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:14:58.804217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:58.804217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:14:58.804217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:14:58.804217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:58.804217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:14:58.804217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:14:58.804217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:14:58.804912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:14:58.804912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:14:58.804912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:14:58.804912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:14:58.804912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:14:58.804912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:58.804912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:58.804912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:14:58.804912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:14:58.804912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:14:58.804912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:14:58.804912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:14:58.804912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:14:58.804912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:14:58.804912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:14:58.804912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:14:58.804912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:14:58.804912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:14:58.804912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:14:58.805471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:14:58.805471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:14:58.805471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:14:58.805471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:14:58.805471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:14:58.805471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:02.957513 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:02.957985 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:02.958157 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:02.959281 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:02.974161 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=240 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:15:02.987936 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:02.988296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:02.989020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:02.989020 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:02.989020 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:02.989020 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:02.989020 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:02.989020 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:02.989020 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:02.989020 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:02.989764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:02.989764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:02.989764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:02.989764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:02.989764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:02.989764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:15:02.989764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:02.989764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:02.989764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:02.989764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:02.989764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:02.989764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:02.990105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:02.990105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:02.990105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:02.990105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:02.990105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:02.990105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:02.990105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:02.990105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:02.990105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:02.990105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:02.990105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:02.990105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:02.990105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:02.990424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:02.990424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:02.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:02.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:02.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:02.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:02.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:02.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:02.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:02.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:02.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:02.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:02.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:02.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:02.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:02.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:02.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:02.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:02.991040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:15:02.991040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:02.991040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:02.991040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:02.991040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:02.991040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:02.991040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:02.991040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:02.991040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:02.991040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:02.991040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:02.991040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:02.991040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:02.991040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:02.991040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:02.991040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:02.991040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:02.991040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:02.991040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:02.991491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:02.991491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:02.991491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:03.734832 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:03.735270 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:03.735456 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:03.736457 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:03.736777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.737603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:03.737603 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:03.737603 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:03.737603 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:03.737603 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:03.737603 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:03.737603 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.737603 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:03.738940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.738940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:03.738940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:03.738940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:03.738940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:03.738940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:15:03.738940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:03.738940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:03.738940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:03.738940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:03.738940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:03.738940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:03.739267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:03.739267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:03.739267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:03.739267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:03.739267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:03.739267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:03.739267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:03.739267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:03.739267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:03.739267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:03.739267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:03.739267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:03.739267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:03.739578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.739578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:03.740738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.740738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:03.740738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:03.740738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:03.740738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:03.740738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:03.740738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:03.740738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:03.740738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:03.740738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:03.740738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:03.740738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:03.740738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:03.740738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:03.740738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:03.740738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:03.741148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:15:03.741148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:03.741148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:03.741148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:03.741148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:03.741148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:03.741148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:03.741148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:03.741148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:03.741148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:03.741148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:03.741148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:03.741148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:03.741148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:03.741148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:03.741148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:03.741148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:03.741148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:03.741148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:03.741620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:03.741620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.741620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:03.784745 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:03.785101 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:03.785275 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:03.785448 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:15:03.785911 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:03.786178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.786922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:03.786922 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:03.786922 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:03.786922 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:03.786922 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:03.786922 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:03.786922 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.786922 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:03.788056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.788056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:15:03.788056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:03.788056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:03.788056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:03.788056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:03.788056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:03.788056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:03.788056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:03.788056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:03.788056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:03.788056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:15:03.788056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:03.788056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:15:03.788056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:15:03.788445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:03.788445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:15:03.788445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:03.788445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:15:03.788445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:15:03.788445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:03.788445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:15:03.788445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.788445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:15:03.789432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.789432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:03.789432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:03.789432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:03.789432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:03.789432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:15:03.789432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:15:03.789432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:15:03.789432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:03.789432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:03.789432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:03.789432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:03.789774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:03.789774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:03.789774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:03.789774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:03.789774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:03.789774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:03.789774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:03.789774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:03.789774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:03.789774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:03.789774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:03.789774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:03.789774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:03.789774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:03.790100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:03.790100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:03.790100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.790100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:03.791054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.791054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:03.791054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:03.791054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:03.791054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:03.791054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:03.791054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:03.791054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:03.791054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:03.791054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:03.791054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:03.791054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:03.791054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:03.791054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:03.791054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:03.791054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:03.791474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:15:03.791474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:15:03.791474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:15:03.791474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:03.791474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:03.791474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:03.791474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:03.791474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:03.791474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:03.791474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:03.791474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:03.791474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:03.791474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:03.791474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:03.791474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:03.791474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:03.791474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:03.791474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:03.791928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:03.791928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:03.791928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:03.791928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:03.791928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:03.791928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.791928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:03.792941 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:03.793233 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:03.793344 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:03.793476 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:03.793886 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:03.794130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.794580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:03.794580 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:03.794580 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:03.794580 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:03.794580 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:03.794580 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:03.794580 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.794580 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:03.795359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.795359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:15:03.795359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:03.795359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:03.795359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:03.795359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:03.795359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:03.795359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:03.795359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:03.795359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:03.795359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:03.795359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:15:03.795359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:03.795359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:15:03.795359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:15:03.795740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:03.795740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:15:03.795740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:03.795740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:15:03.795740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:15:03.795740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:03.795740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:15:03.795740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.795740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:15:03.796276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.796276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:03.796276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:03.796276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:03.796276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:03.796276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:15:03.796276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:15:03.796276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:03.796276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:03.796276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:03.796276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:03.796611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:03.796611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:03.796611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:03.796611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:03.796611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:03.796611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:03.796611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:03.796611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:03.796611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:03.796611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:03.796611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:03.796611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:03.796611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:03.796611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:03.797026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:03.797026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:03.797026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.797026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:03.797437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.797437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:03.797437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:03.797437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:03.797437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:03.797437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:03.797437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:03.797437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:03.797437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:03.797437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:03.797437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:03.797437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:03.797437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:03.797437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:03.797437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:03.797437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:03.797437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:15:03.797937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:15:03.797937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:03.797937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:03.797937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:03.797937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:03.797937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:03.797937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:03.797937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:03.797937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:03.797937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:03.797937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:03.797937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:03.797937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:03.797937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:03.797937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:03.797937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:03.797937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:03.797937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:03.797937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:03.798425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:03.798425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:03.798425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.798425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:03.800071 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:03.800292 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:03.800403 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:03.800549 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:03.800867 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:03.801118 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:03.801242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.801688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:03.801688 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:03.801688 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:03.801688 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:03.801688 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:03.801688 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:03.801688 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.801688 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:03.802513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.802513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:03.802513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:03.802513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:03.802513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:03.802513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:15:03.802513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:03.802513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:03.802513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:03.802513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:03.802513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:03.802513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:03.802901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:03.802901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:03.802901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:03.802901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:03.802901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:03.802901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:03.802901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:03.802901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:03.802901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:03.802901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:03.802901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:03.802901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:03.802901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:03.803229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.803229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:03.803632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.803632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:03.803632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:03.803632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:03.803632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:03.803632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:03.803632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:03.803632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:03.803632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:03.803632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:03.803632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:03.803632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:03.803632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:03.803632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:03.803632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:03.803632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:03.804085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:15:03.804085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:03.804085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:03.804085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:03.804085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:03.804085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:03.804085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:03.804085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:03.804085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:03.804085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:03.804085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:03.804085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:03.804085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:03.804085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:03.804085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:03.804085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:03.804085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:03.804085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:03.804085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:03.804582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:03.804582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:03.804582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:07.958416 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:07.962986 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:07.962986 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:07.962986 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:07.994822 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:07.995224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:07.997821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:07.997821 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:07.997821 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:07.997821 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:07.997821 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:07.997821 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:07.997821 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:07.997821 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:07.997821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:07.997821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:07.997821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:07.997821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:07.997821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:07.997821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:15:07.997821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:07.998300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:07.998300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:07.998300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:07.998300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:07.998300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:07.998300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:07.998300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:07.998300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:07.998300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:07.998300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:07.998300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:07.998300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:07.998300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:07.998300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:07.998582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:07.998582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:07.998582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:07.998582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:07.998582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:07.998582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:07.998582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:07.998582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:07.998582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:07.998582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:07.998582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:07.998582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:07.998582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:07.998582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:07.999007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:07.999007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:07.999007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:07.999007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:07.999007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:07.999007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:07.999007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:07.999007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:07.999007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:15:07.999007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:07.999007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:07.999007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:07.999007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:07.999007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:07.999007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:07.999007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:07.999007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:07.999007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:07.999007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:07.999007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:07.999007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:07.999525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:07.999525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:07.999525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:07.999525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:07.999525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:07.999525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:07.999525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:07.999525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:07.999525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:08.735705 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:08.737139 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:08.737139 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:08.737549 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:08.737895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.741711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:08.741711 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:08.741711 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:08.741711 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:08.741711 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:08.741711 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:08.741711 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.741711 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:08.741711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.741711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:08.741711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:08.741711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:08.741711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:08.741711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:15:08.741711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:08.742508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:08.742508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:08.742508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:08.742508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:08.742508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:08.742508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:08.742508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:08.742508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:08.742508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:08.742508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:08.742508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:08.742508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:08.742508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:08.742508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:08.743131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:08.743131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:08.743131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:08.743131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:08.743131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.743131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:08.743575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.743575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:08.743575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:08.743575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:08.743575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:08.743575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:08.743575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:08.743575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:08.743575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:08.743575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:08.743575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:08.743575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:08.743575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:08.743575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:08.743575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:08.743575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:08.744076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:15:08.744076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:08.744076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:08.744076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:08.744076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:08.744076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:08.744076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:08.744076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:08.744076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:08.744076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:08.744076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:08.744076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:08.744076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:08.744076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:08.744076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:08.744076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:08.744076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:08.744076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:08.744076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:08.744632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:08.744632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.744632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:08.785177 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:08.786231 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:08.786887 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:08.786887 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:15:08.787831 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:08.788139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.788930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:08.788930 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:08.788930 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:08.788930 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:08.788930 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:08.788930 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:08.788930 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.788930 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:08.791961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.791961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:15:08.791961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:08.791961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:08.791961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:08.791961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:08.791961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:08.791961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:08.791961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:08.791961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:08.791961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:08.791961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:15:08.791961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:08.791961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:15:08.791961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:15:08.792422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:08.792422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:15:08.792422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:08.792422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:15:08.792422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:15:08.792422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:08.792422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:15:08.792422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.792422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:15:08.792422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.792422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:08.792422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:08.792422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:08.792422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:08.792882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:15:08.792882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:15:08.792882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:15:08.792882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:08.792882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:08.792882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:08.792882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:08.792882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:08.792882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:08.792882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:08.792882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:08.792882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:08.792882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:08.793277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:08.793277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:08.793277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:08.793277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:08.793277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:08.793277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:08.793277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:08.793277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:08.793277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:08.793277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:08.793277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.793277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:08.794988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.794988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:08.794988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:08.794988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:08.794988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:08.794988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:08.794988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:08.794988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:08.794988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:08.794988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:08.794988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:08.794988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:08.794988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:08.794988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:08.794988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:08.794988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:08.796535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:15:08.796535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:15:08.796535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:15:08.796535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:08.796535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:08.796535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:08.796535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:08.796535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:08.796535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:08.796535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:08.796535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:08.796535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:08.796535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:08.796535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:08.796535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:08.796535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:08.796535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:08.796535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:08.798357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:08.798357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:08.798357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:08.798357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:08.798357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:08.798357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.798357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:08.798357 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:08.798357 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:08.798357 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:08.798991 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:08.798991 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:08.798991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.798991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:08.798991 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:08.798991 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:08.798991 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:08.798991 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:08.798991 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:08.798991 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.798991 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:08.799993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.799993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:15:08.799993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:08.799993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:08.799993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:08.799993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:08.799993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:08.799993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:08.799993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:08.799993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:08.799993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:08.799993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:15:08.799993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:08.799993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:15:08.799993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:15:08.800837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:08.800837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:15:08.800837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:08.800837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:15:08.800837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:15:08.800837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:08.800837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:15:08.800837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.800837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:15:08.801575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.801575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:08.801575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:08.801575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:08.801575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:08.801575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:15:08.801575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:15:08.801575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:08.801575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:08.801575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:08.801575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:08.803153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:08.803153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:08.803153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:08.803153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:08.803153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:08.803153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:08.803153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:08.803153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:08.803153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:08.803153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:08.803153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:08.803153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:08.803153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:08.803153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:08.803615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:08.803615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:08.803615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.803615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:08.803615 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:08.803615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.803615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:08.803615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:08.803615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:08.803615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:08.803615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:08.803615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:08.804079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:08.804079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:08.804079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:08.804079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:08.804079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:08.804079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:08.804079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:08.804079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:08.804079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:08.804079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:15:08.804079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:15:08.804079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:08.804079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:08.804079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:08.804079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:08.804079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:08.804079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:08.804079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:08.804079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:08.804693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:08.804693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:08.804693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:08.804693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:08.804693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:08.804693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:08.804693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:08.804693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:08.804693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:08.804693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:08.804693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:08.804693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:08.804693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.804693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:08.804693 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:08.805183 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:08.805183 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:08.805183 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:08.805183 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:08.805183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.806902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:08.806902 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:08.806902 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:08.806902 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:08.806902 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:08.806902 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:08.806902 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.806902 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:08.806902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.806902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:08.806902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:08.806902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:08.806902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:08.806902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:15:08.806902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:08.807436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:08.807436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:08.807436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:08.807436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:08.807436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:08.807436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:08.807436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:08.807436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:08.807436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:08.807436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:08.807436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:08.807436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:08.807436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:08.807436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:08.807949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:08.807949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:08.807949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:08.807949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:08.807949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.807949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:08.807949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.807949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:08.807949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:08.807949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:08.807949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:08.807949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:08.807949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:08.807949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:08.808463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:08.808463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:08.808463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:08.808463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:08.808463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:08.808463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:08.808463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:08.808463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:08.808463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:15:08.808463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:08.808463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:08.808463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:08.808463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:08.808463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:08.808463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:08.808463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:08.808463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:08.808463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:08.808463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:08.808463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:08.809148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:08.809148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:08.809148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:08.809148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:08.809148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:08.809148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:08.809148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:08.809148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:08.809148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:08.809148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:12.960193 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:12.964751 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:12.964751 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:12.964751 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:13.001607 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:13.002180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.005705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:13.005705 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:13.005705 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:13.005705 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:13.005705 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:13.005705 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:13.005705 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.005705 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:13.005705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.005705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:13.005705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:13.005705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:13.005705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.005705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:15:13.005705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:13.006966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:13.006966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:13.006966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:13.006966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:13.006966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:13.006966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:13.006966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:13.006966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:13.006966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:13.006966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:13.006966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:13.006966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:13.006966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:13.006966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:13.008004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:13.008004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:13.008004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:13.008004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:13.008004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.008004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:13.008004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.008004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:13.008004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:13.008004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:13.008004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:13.008004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:13.008004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.008004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:13.008984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:13.008984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.008984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:13.008984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:13.008984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.008984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:13.008984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:13.008984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.008984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:15:13.008984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:13.008984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:13.008984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:13.008984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:13.008984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:13.008984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:13.008984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:13.008984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:13.008984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:13.008984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:13.008984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:13.008984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:13.010530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:13.010530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:13.010530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:13.010530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:13.010530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:13.010530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:13.010530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:13.010530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.010530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:13.736602 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:13.737508 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:13.737737 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:13.738917 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:13.739263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.740268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:13.740268 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:13.740268 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:13.740268 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:13.740268 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:13.740268 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:13.740268 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.740268 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:13.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:13.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:13.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:13.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:15:13.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:13.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:13.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:13.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:13.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:13.743189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:13.743877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:13.743877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:13.743877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:13.743877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:13.743877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:13.743877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:13.743877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:13.743877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:13.743877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:13.743877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:13.743877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:13.743877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:13.743877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:13.744541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.744541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:13.744944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.744944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:13.744944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:13.744944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:13.744944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:13.744944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:13.744944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.744944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:13.744944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:13.744944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.744944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:13.744944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:13.744944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.744944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:13.744944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:13.744944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.745378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:15:13.745378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:13.745378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:13.745378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:13.745378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:13.745378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:13.745378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:13.745378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:13.745378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:13.745378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:13.745378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:13.745378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:13.745378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:13.745378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:13.745378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:13.745378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:13.745378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:13.745378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:13.745378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:13.745860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:13.745860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.745860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:13.786010 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:13.786610 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:13.786865 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:13.787113 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:15:13.787752 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:13.788054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.789190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:13.789190 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:13.789190 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:13.789190 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:13.789190 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:13.789190 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:13.789190 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.789190 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:13.790917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.790917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:15:13.790917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:13.790917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:13.790917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:13.790917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:13.790917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:13.790917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:13.790917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:13.790917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:13.790917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:13.790917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:15:13.790917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:13.790917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:15:13.790917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:15:13.791320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:13.791320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:15:13.791320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:13.791320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:15:13.791320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:15:13.791320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:13.791320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:15:13.791320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.791320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:15:13.792691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.792691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:13.792691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:13.792691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:13.792691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.792691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:15:13.792691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:15:13.792691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:15:13.792691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:13.792691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:13.792691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:13.792691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:13.793030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:13.793030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:13.793030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:13.793030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:13.793030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:13.793030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:13.793030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:13.793030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:13.793030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:13.793030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:13.793030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:13.793030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:13.793030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:13.793030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:13.793360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:13.793360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:13.793360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.793360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:13.794576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.794576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:13.794576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:13.794576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:13.794576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:13.794576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:13.794576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.794576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:13.794576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:13.794576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.794576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:13.794576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:13.794576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.794576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:13.794576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:13.794576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.795020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:15:13.795020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:15:13.795020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:15:13.795020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:13.795020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:13.795020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:13.795020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:13.795020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:13.795020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:13.795020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:13.795020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:13.795020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:13.795020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:13.795020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:13.795020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:13.795020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:13.795020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:13.795020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:13.795438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:13.795438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:13.795438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:13.795438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:13.795438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:13.795438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.795438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:13.795620 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:13.795966 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:13.796283 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:13.796283 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:13.796636 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:13.797017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.797798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:13.797798 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:13.797798 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:13.797798 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:13.797798 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:13.797798 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:13.797798 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.797798 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:13.798579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.798579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:15:13.798579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:13.798579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:13.798579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:13.798579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:13.798579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:13.798579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:13.798579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:13.798579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:13.798579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:13.798579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:15:13.798579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:13.798579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:15:13.798579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:15:13.799646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:13.799646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:15:13.799646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:13.799646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:15:13.799646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:15:13.799646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:13.799646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:15:13.799646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.799646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:15:13.799646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.799646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:13.799646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:13.799646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:13.799646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.800174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:15:13.800174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:15:13.800174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:13.800174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:13.800174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:13.800174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:13.800174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:13.800174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:13.800174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:13.800174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:13.800174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:13.800174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:13.800174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:13.800490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:13.800490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:13.800490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:13.800490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:13.800490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:13.800490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:13.800490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:13.800490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:13.800490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:13.800490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.800490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:13.800768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.800768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:13.800768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:13.800768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:13.800768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:13.800768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:13.800768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.800768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:13.800768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:13.800768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.800768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:13.800768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:13.800768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.800768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:13.800768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:13.800768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.800768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:15:13.801186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:15:13.801186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:13.801186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:13.801186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:13.801186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:13.801186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:13.801186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:13.801186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:13.801186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:13.801186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:13.801186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:13.801186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:13.801186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:13.801186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:13.801186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:13.801186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:13.801186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:13.801186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:13.801186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:13.801654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:13.801654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:13.801654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.801654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:13.802877 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:13.803219 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:13.803331 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:13.803492 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:13.804001 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:13.804342 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:13.804521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.805270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:13.805270 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:13.805270 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:13.805270 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:13.805270 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:13.805270 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:13.805270 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.805270 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:13.806159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.806159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:13.806159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:13.806159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:13.806159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.806159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:15:13.806159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:13.806159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:13.806159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:13.806159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:13.806159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:13.806159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:13.806833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:13.806833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:13.806833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:13.806833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:13.806833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:13.806833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:13.806833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:13.806833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:13.806833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:13.806833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:13.806833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:13.806833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:13.806833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:13.807493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.807493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:13.807493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.807493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:13.807493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:13.807493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:13.807493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:13.807493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:13.807493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.807493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:13.807493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:13.807493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.807493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:13.807493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:13.807493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.808321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:13.808321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:13.808321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:13.808321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:15:13.808321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:13.808321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:13.808321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:13.808321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:13.808321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:13.808321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:13.808321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:13.808321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:13.808321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:13.808321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:13.808321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:13.808321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:13.808321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:13.808321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:13.808321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:13.809296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:13.809296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:13.809296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:13.809296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:13.809296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:13.809296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:17.962051 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:17.963230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:17.963230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:17.964237 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:17.993665 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:17.994196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:17.995327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:17.995327 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:17.995327 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:17.995327 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:17.995327 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:17.995327 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:17.995327 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:17.995327 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:17.996220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:17.996220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:17.996220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:17.996220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:17.996220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:17.996220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:15:17.996220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:17.996220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:17.996220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:17.996220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:17.996220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:17.996220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:17.996969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:17.996969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:17.996969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:17.996969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:17.996969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:17.996969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:17.996969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:17.996969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:17.996969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:17.996969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:17.996969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:17.996969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:17.996969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:17.997588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:17.997588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:17.997588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:17.997588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:17.997588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:17.997588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:17.997588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:17.997588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:17.997588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:17.997588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:17.997588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:17.997588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:17.997588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:17.997588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:17.997588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:17.998662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:17.998662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:17.998662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:17.998662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:15:17.998662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:17.998662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:17.998662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:17.998662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:17.998662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:17.998662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:17.998662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:17.998662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:17.998662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:17.998662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:17.998662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:17.998662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:17.998662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:17.998662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:17.998662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:18.000311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:18.000311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:18.000311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:18.000311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:18.000311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.000311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:18.738995 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:18.739449 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:18.739449 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:18.740786 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:18.742537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.742694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:18.742694 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:18.742694 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:18.742694 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:18.742694 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:18.742694 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:18.742694 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.742694 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:18.743859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.743859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:18.743859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:18.743859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:18.743859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:18.743859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:15:18.743859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:18.743859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:18.743859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:18.743859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:18.743859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:18.743859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:18.744244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:18.744244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:18.744244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:18.744244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:18.744244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:18.744244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:18.744244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:18.744244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:18.744244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:18.744244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:18.744244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:18.744244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:18.744244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:18.744612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.744612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:18.744932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.744932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:18.744932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:18.744932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:18.744932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:18.744932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:18.744932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:18.744932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:18.744932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:18.744932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:18.744932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:18.744932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:18.744932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:18.744932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:18.744932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:18.744932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:18.745402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:15:18.745402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:18.745402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:18.745402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:18.745402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:18.745402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:18.745402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:18.745402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:18.745402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:18.745402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:18.745402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:18.745402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:18.745402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:18.745402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:18.745402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:18.745402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:18.745402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:18.745402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:18.745402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:18.745882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:18.745882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.745882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:18.787371 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:18.787866 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:18.788065 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:18.788269 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:15:18.789224 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:18.789605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.790595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:18.790595 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:18.790595 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:18.790595 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:18.790595 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:18.790595 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:18.790595 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.790595 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:18.792186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.792186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:15:18.792186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:18.792186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:18.792186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:18.792186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:18.792186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:18.792186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:18.792186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:18.792186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:18.792186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:18.792186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:15:18.792186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:18.792186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:15:18.792186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:15:18.792660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:18.792660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:15:18.792660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:18.792660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:15:18.792660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:15:18.792660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:18.792660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:15:18.792660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.792660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:15:18.793609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.793609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:18.793609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:18.793609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:18.793609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:18.793609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:15:18.793609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:15:18.793609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:15:18.793609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:18.793609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:18.793609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:18.793609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:18.794008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:18.794008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:18.794008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:18.794008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:18.794008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:18.794008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:18.794008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:18.794008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:18.794008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:18.794008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:18.794008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:18.794008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:18.794008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:18.794008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:18.794413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:18.794413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:18.794413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.794413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:18.795705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.795705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:18.795705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:18.795705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:18.795705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:18.795705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:18.795705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:18.795705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:18.795705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:18.795705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:18.795705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:18.795705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:18.795705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:18.795705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:18.795705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:18.795705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:18.796468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:15:18.796468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:15:18.796468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:15:18.796468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:18.796468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:18.796468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:18.796468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:18.796468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:18.796468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:18.796468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:18.796468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:18.796468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:18.796468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:18.796468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:18.796468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:18.796468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:18.796468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:18.796468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:18.797013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:18.797013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:18.797013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:18.797013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:18.797013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:18.797013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.797013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:18.797013 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:18.797013 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:18.797013 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:18.797288 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:18.797458 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:18.797902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.798768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:18.798768 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:18.798768 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:18.798768 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:18.798768 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:18.798768 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:18.798768 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.798768 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:18.799532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.799532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:15:18.799532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:18.799532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:18.799532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:18.799532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:18.799532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:18.799532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:18.799532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:18.799532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:18.799532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:18.799532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:15:18.799532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:18.799532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:15:18.799532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:15:18.799974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:18.799974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:15:18.799974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:18.799974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:15:18.799974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:15:18.799974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:18.799974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:15:18.799974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.799974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:15:18.800347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.800347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:18.800347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:18.800347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:18.800347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:18.800347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:15:18.800347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:15:18.800347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:18.800347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:18.800347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:18.800347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:18.800902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:18.800902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:18.800902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:18.800902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:18.800902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:18.800902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:18.800902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:18.800902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:18.800902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:18.800902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:18.800902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:18.800902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:18.800902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:18.800902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:18.801258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:18.801258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:18.801258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.801258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:18.801624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.801624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:18.801624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:18.801624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:18.801624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:18.801624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:18.801624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:18.801624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:18.801624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:18.801624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:18.801624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:18.801624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:18.801624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:18.801624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:18.801624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:18.801624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:18.801624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:15:18.802104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:15:18.802104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:18.802104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:18.802104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:18.802104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:18.802104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:18.802104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:18.802104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:18.802104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:18.802104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:18.802104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:18.802104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:18.802104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:18.802104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:18.802104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:18.802104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:18.802104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:18.802104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:18.802104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:18.802901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:18.802901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:18.802901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.802901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:18.803371 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:18.803502 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:18.803608 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:18.803783 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:18.804325 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:18.804772 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:18.804973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.805779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:18.805779 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:18.805779 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:18.805779 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:18.805779 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:18.805779 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:18.805779 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.805779 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:18.806591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.806591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:18.806591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:18.806591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:18.806591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:18.806591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:15:18.806591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:18.806591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:18.806591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:18.806591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:18.806591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:18.806591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:18.806951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:18.806951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:18.806951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:18.806951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:18.806951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:18.806951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:18.806951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:18.806951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:18.806951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:18.806951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:18.806951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:18.806951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:18.806951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:18.807315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.807315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:18.807540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.807540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:18.807540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:18.807540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:18.807540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:18.807540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:18.807540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:18.807540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:18.807540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:18.807540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:18.807540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:18.807540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:18.807540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:18.807540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:18.807540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:18.807540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:18.808188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:15:18.808188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:18.808188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:18.808188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:18.808188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:18.808188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:18.808188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:18.808188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:18.808188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:18.808188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:18.808188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:18.808188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:18.808188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:18.808188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:18.808188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:18.808188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:18.808188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:18.808188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:18.808188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:18.809082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:18.809082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:18.809082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:22.962655 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:22.963496 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:22.963730 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:22.965146 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:23.005036 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:23.005529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.006762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:23.006762 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:23.006762 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:23.006762 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:23.006762 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:23.006762 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:23.006762 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.006762 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:23.008081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.008081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:23.008081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:23.008081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:23.008081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.008081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:15:23.008081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:23.008081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:23.008081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:23.008081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:23.008081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:23.008081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:23.008428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:23.008428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:23.008428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:23.008428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:23.008428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:23.008428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:23.008428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:23.008428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:23.008428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:23.008428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:23.008428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:23.008428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:23.008428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:23.008836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.008836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:23.009315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.009315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:23.009315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:23.009315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:23.009315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:23.009315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:23.009315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.009315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:23.009315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:23.009315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.009315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:23.009315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:23.009315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.009315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:23.009315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:23.009315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.009776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:15:23.009776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:23.009776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:23.009776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:23.009776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:23.009776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:23.009776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:23.009776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:23.009776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:23.009776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:23.009776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:23.009776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:23.009776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:23.009776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:23.009776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:23.009776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:23.009776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:23.009776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:23.009776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:23.010238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:23.010238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.010238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:23.739409 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:23.740285 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:23.740535 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:23.742535 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:23.743133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.745130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:23.745130 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:23.745130 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:23.745130 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:23.745130 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:23.745130 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:23.745130 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.745130 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:23.746943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.746943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:23.746943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:23.746943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:23.746943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.746943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:15:23.746943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:23.746943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:23.746943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:23.746943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:23.746943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:23.746943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:23.749343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:23.749343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:23.749343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:23.749343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:23.749343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:23.749343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:23.749343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:23.749343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:23.749343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:23.749343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:23.749343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:23.749343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:23.749343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:23.749931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.749931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:23.749931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.749931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:23.749931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:23.749931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:23.749931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:23.749931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:23.749931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.749931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:23.749931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:23.749931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.749931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:23.749931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:23.749931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.750987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:23.750987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:23.750987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.750987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:15:23.750987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:23.750987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:23.750987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:23.750987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:23.750987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:23.750987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:23.750987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:23.750987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:23.750987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:23.750987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:23.750987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:23.750987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:23.750987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:23.750987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:23.750987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:23.752359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:23.752359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:23.752359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:23.752359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:23.752359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.752359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:23.787849 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:23.789442 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:23.789699 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:23.789905 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:15:23.790867 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:23.791272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.792293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:23.792293 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:23.792293 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:23.792293 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:23.792293 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:23.792293 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:23.792293 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.792293 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:23.793635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.793635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:15:23.793635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:23.793635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:23.793635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:23.793635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:23.793635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:23.793635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:23.793635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:23.793635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:23.793635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:23.793635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:15:23.793635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:23.793635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:15:23.793635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:15:23.794102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:23.794102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:15:23.794102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:23.794102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:15:23.794102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:15:23.794102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:23.794102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:15:23.794102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.794102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:15:23.794952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.794952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:23.794952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:23.794952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:23.794952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.794952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:15:23.794952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:15:23.794952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:15:23.794952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:23.794952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:23.794952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:23.794952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:23.795304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:23.795304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:23.795304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:23.795304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:23.795304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:23.795304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:23.795304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:23.795304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:23.795304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:23.795304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:23.795304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:23.795304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:23.795304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:23.795304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:23.795657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:23.795657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:23.795657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.795657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:23.796036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.796036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:23.796036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:23.796036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:23.796036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:23.796036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:23.796036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.796036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:23.796036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:23.796036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.796036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:23.796036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:23.796036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.796036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:23.796036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:23.796036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.796435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:15:23.796435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:15:23.796435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:15:23.796435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:23.796435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:23.796435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:23.796435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:23.796435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:23.796435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:23.796435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:23.796435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:23.796435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:23.796435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:23.796435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:23.796435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:23.796435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:23.796435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:23.796435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:23.796884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:23.796884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:23.796884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:23.796884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:23.796884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:23.796884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.796884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:23.796884 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:23.796884 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:23.797119 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:23.797119 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:23.797468 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:23.797768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.798254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:23.798254 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:23.798254 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:23.798254 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:23.798254 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:23.798254 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:23.798254 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.798254 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:23.799210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.799210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:15:23.799210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:23.799210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:23.799210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:23.799210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:23.799210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:23.799210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:23.799210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:23.799210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:23.799210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:23.799210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:15:23.799210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:23.799210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:15:23.799210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:15:23.799593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:23.799593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:15:23.799593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:23.799593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:15:23.799593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:15:23.799593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:23.799593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:15:23.799593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.799593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:15:23.800026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.800026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:23.800026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:23.800026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:23.800026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.800026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:15:23.800026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:15:23.800026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:23.800026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:23.800026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:23.800026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:23.800653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:23.800653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:23.800653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:23.800653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:23.800653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:23.800653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:23.800653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:23.800653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:23.800653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:23.800653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:23.800653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:23.800653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:23.800653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:23.800653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:23.800999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:23.800999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:23.800999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.800999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:23.801105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.801105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:23.801105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:23.801105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:23.801105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:23.801105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:23.801105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.801105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:23.801105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:23.801105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.801105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:23.801105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:23.801105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.801105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:23.801105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:23.801105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.801105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:15:23.801533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:15:23.801533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:23.801533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:23.801533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:23.801533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:23.801533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:23.801533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:23.801533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:23.801533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:23.801533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:23.801533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:23.801533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:23.801533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:23.801533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:23.801533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:23.801533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:23.801533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:23.801533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:23.801533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:23.802044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:23.802044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:23.802044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.802044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:23.803906 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:23.804219 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:23.804330 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:23.804481 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:23.804856 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:23.805089 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:23.805217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.805756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:23.805756 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:23.805756 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:23.805756 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:23.805756 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:23.805756 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:23.805756 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.805756 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:23.806543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.806543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:23.806543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:23.806543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:23.806543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.806543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:15:23.806543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:23.806543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:23.806543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:23.806543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:23.806543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:23.806543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:23.806892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:23.806892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:23.806892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:23.806892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:23.806892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:23.806892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:23.806892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:23.806892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:23.806892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:23.806892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:23.806892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:23.806892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:23.806892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:23.807178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.807178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:23.807499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.807499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:23.807499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:23.807499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:23.807499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:23.807499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:23.807499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.807499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:23.807499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:23.807499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.807499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:23.807499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:23.807499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.807499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:23.807499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:23.807499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:23.807890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:15:23.807890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:23.807890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:23.807890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:23.807890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:23.807890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:23.807890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:23.807890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:23.807890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:23.807890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:23.807890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:23.807890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:23.807890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:23.807890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:23.807890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:23.807890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:23.807890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:23.807890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:23.807890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:23.808340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:23.808340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:23.808340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:27.963452 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:27.964264 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:27.964264 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:27.964620 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:27.995915 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:27.996027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:27.996471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:27.996471 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:27.996471 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:27.996471 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:27.996471 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:27.996471 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:27.996471 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:27.996471 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:27.997148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:27.997148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:27.997148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:27.997148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:27.997148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:27.997148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:15:27.997148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:27.997148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:27.997148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:27.997148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:27.997148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:27.997148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:27.997660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:27.997660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:27.997660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:27.997660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:27.997660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:27.997660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:27.997660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:27.997660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:27.997660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:27.997660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:27.997660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:27.997660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:27.997660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:27.998239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:27.998239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:27.998239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:27.998239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:27.998239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:27.998239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:27.998239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:27.998239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:27.998239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:27.998239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:27.998239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:27.998239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:27.998239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:27.998239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:27.998239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:27.999043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:27.999043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:27.999043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:27.999043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:15:27.999043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:27.999043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:27.999043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:27.999043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:27.999043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:27.999043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:27.999043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:27.999043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:27.999043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:27.999043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:27.999043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:27.999043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:27.999043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:27.999043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:27.999043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:27.999823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:27.999823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:27.999823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:27.999823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:27.999823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:27.999823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:28.739895 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:28.740543 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:28.740860 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:28.741738 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:28.741993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.743290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:28.743290 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:28.743290 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:28.743290 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:28.743290 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:28.743290 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:28.743290 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.743290 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:28.745111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.745111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:28.745111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:28.745111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:28.745111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:28.745111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:15:28.745111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:28.745111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:28.745111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:28.745111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:28.745111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:28.745111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:28.745665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:28.745665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:28.745665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:28.745665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:28.745665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:28.745665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:28.745665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:28.745665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:28.745665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:28.745665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:28.745665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:28.745665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:28.745665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:28.746190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.746190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:28.747350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.747350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:28.747350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:28.747350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:28.747350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:28.747350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:28.747350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:28.747350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:28.747350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:28.747350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:28.747350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:28.747350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:28.747350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:28.747350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:28.747350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:28.747350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:28.747975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:15:28.747975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:28.747975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:28.747975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:28.747975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:28.747975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:28.747975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:28.747975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:28.747975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:28.747975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:28.747975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:28.747975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:28.747975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:28.747975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:28.747975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:28.747975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:28.747975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:28.747975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:28.747975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:28.748452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:28.748452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.748452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:28.789398 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:28.790011 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:28.790215 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:28.790397 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:15:28.790967 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:28.791282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.792116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:28.792116 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:28.792116 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:28.792116 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:28.792116 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:28.792116 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:28.792116 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.792116 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:28.793803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.793803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:15:28.793803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:28.793803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:28.793803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:28.793803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:28.793803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:28.793803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:28.793803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:28.793803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:28.793803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:28.793803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:15:28.793803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:28.793803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:15:28.793803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:15:28.794493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:28.794493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:15:28.794493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:28.794493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:15:28.794493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:15:28.794493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:28.794493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:15:28.794493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.794493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:15:28.796194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.796194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:28.796194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:28.796194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:28.796194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:28.796194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:15:28.796194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:15:28.796194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:15:28.796194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:28.796194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:28.796194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:28.796194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:28.796775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:28.796775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:28.796775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:28.796775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:28.796775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:28.796775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:28.796775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:28.796775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:28.796775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:28.796775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:28.796775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:28.796775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:28.796775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:28.796775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:28.797324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:28.797324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:28.797324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.797324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:28.798482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.798482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:28.798482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:28.798482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:28.798482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:28.798482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:28.798482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:28.798482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:28.798482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:28.798482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:28.798482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:28.798482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:28.798482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:28.798482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:28.798482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:28.798482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:28.799047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:15:28.799047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:15:28.799047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:15:28.799047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:28.799047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:28.799047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:28.799047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:28.799047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:28.799047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:28.799047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:28.799047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:28.799047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:28.799047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:28.799047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:28.799047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:28.799047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:28.799047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:28.799047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:28.799528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:28.799528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:28.799528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:28.799528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:28.799528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:28.799528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.799528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:28.799528 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:28.799528 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:28.799528 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:28.799845 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:28.800327 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:28.800327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.800850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:28.800850 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:28.800850 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:28.800850 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:28.800850 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:28.800850 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:28.800850 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.800850 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:28.801782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.801782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:15:28.801782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:28.801782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:28.801782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:28.801782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:28.801782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:28.801782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:28.801782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:28.801782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:28.801782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:28.801782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:15:28.801782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:28.801782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:15:28.801782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:15:28.802718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:28.802718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:15:28.802718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:28.802718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:15:28.802718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:15:28.802718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:28.802718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:15:28.802718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.802718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:15:28.802718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.802718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:28.802718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:28.802718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:28.802718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:28.803786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:15:28.803786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:15:28.803786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:28.803786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:28.803786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:28.803786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:28.803786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:28.803786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:28.803786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:28.803786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:28.803786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:28.803786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:28.803786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:28.804406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:28.804406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:28.804406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:28.804406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:28.804406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:28.804406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:28.804406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:28.804406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:28.804406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:28.804406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.804406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:28.804406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.804732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:28.804732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:28.804732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:28.804732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:28.804732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:28.804732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:28.804732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:28.804732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:28.804732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:28.804732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:28.804732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:28.804732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:28.804732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:28.804732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:28.804732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:28.804732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:15:28.804732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:15:28.804732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:28.804732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:28.805207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:28.805207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:28.805207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:28.805207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:28.805207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:28.805207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:28.805207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:28.805207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:28.805207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:28.805207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:28.805207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:28.805207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:28.805207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:28.805207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:28.805207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:28.805207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:28.805207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:28.805207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:28.805207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.805207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:28.806195 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:28.806195 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:28.806195 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:28.806195 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:28.806195 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:28.806195 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:28.806590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.806590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:28.806590 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:28.806590 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:28.806590 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:28.806590 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:28.806590 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:28.806590 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.806590 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:28.807203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.807203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:28.807203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:28.807203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:28.807203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:28.807203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:15:28.807203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:28.807203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:28.807203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:28.807203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:28.807203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:28.807203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:28.807918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:28.807918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:28.807918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:28.807918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:28.807918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:28.807918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:28.807918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:28.807918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:28.807918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:28.807918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:28.807918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:28.807918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:28.807918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:28.808389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.808389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:28.808389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.808389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:28.808389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:28.808389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:28.808389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:28.808389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:28.808389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:28.808389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:28.808389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:28.808389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:28.808389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:28.808389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:28.808389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:28.808940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:28.808940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:28.808940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:28.808940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:15:28.808940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:28.808940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:28.808940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:28.808940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:28.808940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:28.808940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:28.808940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:28.808940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:28.808940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:28.808940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:28.808940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:28.808940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:28.808940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:28.808940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:28.808940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:28.809258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:28.809258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:28.809258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:28.809258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:28.809258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:28.809258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:32.964139 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:32.964914 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:32.964914 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:32.965297 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:32.989193 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:32.989353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:32.989800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:32.989800 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:32.989800 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:32.989800 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:32.989800 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:32.989800 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:32.989800 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:32.989800 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:32.990489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.025 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:32.990489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:32.990489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:32.990489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:32.990489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:32.990489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:15:32.990489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:32.990489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:32.990489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:32.990489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:32.990489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:32.990489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:32.991482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:32.991482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:32.991482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:32.991482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:32.991482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:32.991482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:32.991482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:32.991482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:32.991482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:32.991482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:32.991482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:32.991482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:32.991482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:32.992051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:32.992051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:32.992051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:32.992051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:32.992051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:32.992051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:32.992051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:32.992051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:32.992051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:32.992051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:32.992051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:32.992051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:32.992051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:32.992051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:32.992051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:32.992864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:32.992864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:32.992864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:32.992864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:15:32.992864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:32.992864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:32.992864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:32.992864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:32.992864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:32.992864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:32.992864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:32.992864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:32.992864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:32.992864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:32.992864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:32.992864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:32.992864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:32.992864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:32.992864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:32.994607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:32.994607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:32.994607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:32.994607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:32.994607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:32.994607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:33.740654 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:33.741284 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:33.741581 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:33.742435 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:33.742744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.743876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:33.743876 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:33.743876 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:33.743876 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:33.743876 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:33.743876 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:33.743876 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.743876 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:33.745564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.745564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:33.745564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:33.745564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:33.745564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:33.745564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:15:33.745564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:33.745564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:33.745564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:33.745564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:33.745564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:33.745564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:33.746241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:33.746241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:33.746241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:33.746241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:33.746241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:33.746241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:33.746241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:33.746241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:33.746241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:33.746241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:33.746241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:33.746241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:33.746241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:33.746767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.746767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:33.747277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.747277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:33.747277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:33.747277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:33.747277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:33.747277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:33.747277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:33.747277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:33.747277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:33.747277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:33.747277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:33.747277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:33.747277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:33.747277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:33.747277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:33.747277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:33.750169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:15:33.750169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:33.750169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:33.750169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:33.750169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:33.750169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:33.750169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:33.750169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:33.750169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:33.750169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:33.750169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:33.750169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:33.750169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:33.750169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:33.750169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:33.750169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:33.750169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:33.750169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:33.750169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:33.750799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:33.750799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.750799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:33.789974 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:33.790284 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:33.790476 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:33.791964 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:15:33.791964 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:33.791964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.792324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:33.792324 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:33.792324 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:33.792324 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:33.792324 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:33.792324 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:33.792324 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.792324 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:33.793563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.793563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:15:33.793563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:33.793563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:33.793563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:33.793563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:33.793563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:33.793563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:33.793563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:33.793563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:33.793563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:33.793563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:15:33.793563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:33.793563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:15:33.793563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:15:33.794217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:33.794217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:15:33.794217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:33.794217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:15:33.794217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:15:33.794217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:33.794217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:15:33.794217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.794217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:15:33.795069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.795069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:33.795069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:33.795069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:33.795069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:33.795069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:15:33.795069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:15:33.795069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:15:33.795069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:33.795069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:33.795069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:33.795069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:33.795607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:33.795607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:33.795607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:33.795607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:33.795607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:33.795607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:33.795607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:33.795607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:33.795607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:33.795607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:33.795607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:33.795607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:33.795607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:33.795607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:33.796233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:33.796233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:33.796233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.796233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:33.796842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.796842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:33.796842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:33.796842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:33.796842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:33.796842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:33.796842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:33.796842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:33.796842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:33.796842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:33.796842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:33.796842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:33.796842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:33.796842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:33.796842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:33.796842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:33.797793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:15:33.797793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:15:33.797793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:15:33.797793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:33.797793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:33.797793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:33.797793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:33.797793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:33.797793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:33.797793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:33.797793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:33.797793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:33.797793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:33.797793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:33.797793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:33.797793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:33.797793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:33.797793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:33.798321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:33.798321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:33.798321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:33.798321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:33.798321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:33.798321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.798321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:33.799136 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:33.799342 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:33.799448 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:33.799545 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:33.799904 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:33.800126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.800438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:33.800438 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:33.800438 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:33.800438 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:33.800438 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:33.800438 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:33.800438 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.800438 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:33.800956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.800956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:15:33.800956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:33.800956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:33.800956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:33.800956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:33.800956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:33.800956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:33.800956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:33.800956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:33.800956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:33.800956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:15:33.800956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:33.800956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:15:33.800956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:15:33.801232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:33.801232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:15:33.801232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:33.801232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:15:33.801232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:15:33.801232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:33.801232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:15:33.801232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.801232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:15:33.801701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.801701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:33.801701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:33.801701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:33.801701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:33.801701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:15:33.801701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:15:33.801701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:33.801701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:33.801701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:33.801701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:33.801913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:33.801913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:33.801913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:33.801913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:33.801913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:33.801913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:33.801913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:33.801913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:33.801913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:33.801913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:33.801913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:33.801913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:33.801913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:33.801913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:33.802160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:33.802160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:33.802160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.802160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:33.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:33.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:33.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:33.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:33.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:33.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:33.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:33.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:33.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:33.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:33.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:33.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:33.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:33.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:33.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:33.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:15:33.802897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:15:33.802897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:33.802897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:33.802897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:33.802897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:33.802897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:33.802897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:33.802897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:33.802897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:33.802897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:33.802897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:33.802897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:33.802897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:33.802897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:33.802897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:33.802897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:33.802897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:33.802897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:33.802897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:33.803737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:33.803737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:33.803737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.803737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:33.804730 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:33.804964 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:33.805039 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:33.805162 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:33.805358 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:33.805614 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:33.805742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.806131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:33.806131 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:33.806131 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:33.806131 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:33.806131 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:33.806131 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:33.806131 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.806131 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:33.806969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.806969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:33.806969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:33.806969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:33.806969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:33.806969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:15:33.806969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:33.806969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:33.806969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:33.806969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:33.806969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:33.806969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:33.807310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:33.807310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:33.807310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:33.807310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:33.807310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:33.807310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:33.807310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:33.807310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:33.807310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:33.807310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:33.807310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:33.807310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:33.807310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:33.807629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.807629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:33.808052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.808052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:33.808052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:33.808052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:33.808052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:33.808052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:33.808052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:33.808052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:33.808052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:33.808052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:33.808052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:33.808052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:33.808052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:33.808052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:33.808052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:33.808052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:33.809126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:15:33.809126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:33.809126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:33.809126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:33.809126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:33.809126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:33.809126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:33.809126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:33.809126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:33.809126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:33.809126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:33.809126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:33.809126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:33.809126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:33.809126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:33.809126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:33.809126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:33.809126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:33.809126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:33.810069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:33.810069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:33.810069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:37.964809 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:37.965526 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:37.965526 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:37.966384 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:38.004301 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:38.004561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.005805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:38.005805 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:38.005805 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:38.005805 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:38.005805 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:38.005805 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:38.005805 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.005805 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:38.008059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.008059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:38.008059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:38.008059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:38.008059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.008059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:15:38.008059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:38.008059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:38.008059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:38.008059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:38.008059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:38.008059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:38.008721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:38.008721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:38.008721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:38.008721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:38.008721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:38.008721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:38.008721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:38.008721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:38.008721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:38.008721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:38.008721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:38.008721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:38.008721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:38.009347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.009347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:38.009600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.009600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:38.009600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:38.009600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:38.009600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:38.009600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:38.009600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.009600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:38.009600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:38.009600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.009600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:38.009600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:38.009600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.009600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:38.009600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:38.009600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.010397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:15:38.010397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:38.010397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:38.010397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:38.010397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:38.010397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:38.010397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:38.010397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:38.010397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:38.010397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:38.010397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:38.010397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:38.010397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:38.010397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:38.010397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:38.010397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:38.010397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:38.010397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:38.010397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:38.011294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:38.011294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.011294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:38.741211 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:38.741891 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:38.742199 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:38.743613 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:38.743927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.744708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:38.744708 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:38.744708 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:38.744708 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:38.744708 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:38.744708 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:38.744708 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.744708 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:38.745953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.745953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:38.745953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:38.745953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:38.745953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.745953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:15:38.745953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:38.745953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:38.745953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:38.745953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:38.745953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:38.745953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:38.746811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:38.746811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:38.746811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:38.746811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:38.746811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:38.746811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:38.746811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:38.746811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:38.746811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:38.746811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:38.746811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:38.746811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:38.746811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:38.747409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.747409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:38.747541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.747541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:38.747541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:38.747541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:38.747541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:38.747541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:38.747541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.747541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:38.747541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:38.747541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.747541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:38.747541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:38.747541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.747541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:38.747541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:38.747541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:15:38.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:38.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:38.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:38.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:38.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:38.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:38.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:38.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:38.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:38.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:38.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:38.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:38.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:38.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:38.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:38.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:38.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:38.748368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:38.749163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:38.749163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.749163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:38.790454 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:38.791038 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:38.791235 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:38.791415 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:15:38.793250 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:38.793549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.794477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:38.794477 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:38.794477 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:38.794477 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:38.794477 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:38.794477 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:38.794477 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.794477 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:38.795637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.795637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:15:38.795637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:38.795637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:38.795637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:38.795637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:38.795637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:38.795637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:38.795637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:38.795637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:38.795637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:38.795637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:15:38.795637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:38.795637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:15:38.795637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:15:38.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:38.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:15:38.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:38.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:15:38.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:15:38.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:38.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:15:38.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:15:38.797049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.797049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:38.797049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:38.797049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:38.797049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.797049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:15:38.797049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:15:38.797049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:15:38.797049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:38.797049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:38.797049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:38.797049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:38.797513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:38.797513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:38.797513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:38.797513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:38.797513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:38.797513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:38.797513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:38.797513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:38.797513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:38.797513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:38.797513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:38.797513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:38.797513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:38.797513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:38.797916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:38.797916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:38.797916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.797916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:38.798196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.798196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:38.798196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:38.798196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:38.798196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:38.798196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:38.798196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.798196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:38.798196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:38.798196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.798196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:38.798196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:38.798196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.798196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:38.798196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:38.798196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.798915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:15:38.798915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:15:38.798915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:15:38.798915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:38.798915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:38.798915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:38.798915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:38.798915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:38.798915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:38.798915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:38.798915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:38.798915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:38.798915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:38.798915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:38.798915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:38.798915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:38.798915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:38.798915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:38.799629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:38.799629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:38.799629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:38.799629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:38.799629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:38.799629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.799629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:38.799629 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:38.799984 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:38.799984 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:38.800121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:38.800438 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:38.800691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.801182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:38.801182 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:38.801182 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:38.801182 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:38.801182 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:38.801182 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:38.801182 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.801182 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:38.801976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.801976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:15:38.801976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:38.801976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:38.801976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:38.801976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:38.801976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:38.801976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:38.801976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:38.801976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:38.801976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:38.801976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:15:38.801976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:38.801976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:15:38.801976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:15:38.802408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:38.802408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:15:38.802408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:38.802408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:15:38.802408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:15:38.802408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:38.802408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:15:38.802408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.802408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:15:38.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:38.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:38.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:38.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:15:38.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:15:38.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:38.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:38.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:38.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:38.803343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:38.803343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:38.803343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:38.803343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:38.803343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:38.803343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:38.803343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:38.803343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:38.803343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:38.803343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:38.803343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:38.803343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:38.803343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:38.803343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:38.803766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:38.803766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:38.803766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.803766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:38.804326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.804326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:38.804326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:38.804326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:38.804326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:38.804326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:38.804326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.804326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:38.804326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:38.804326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.804326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:38.804326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:38.804326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.804326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:38.804326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:38.804326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.804326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:15:38.804861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:15:38.804861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:38.804861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:38.804861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:38.804861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:38.804861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:38.804861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:38.804861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:38.804861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:38.804861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:38.804861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:38.804861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:38.804861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:38.804861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:38.804861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:38.804861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:38.804861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:38.804861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:38.804861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:38.805384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:38.805384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:38.805384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.805384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:38.805384 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:38.805384 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:38.805653 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:38.805888 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:38.806141 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:38.806361 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:38.806467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.807001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:38.807001 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:38.807001 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:38.807001 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:38.807001 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:38.807001 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:38.807001 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.807001 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:38.807703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.807703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:38.807703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:38.807703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:38.807703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.807703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:15:38.807703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:38.807703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:38.807703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:38.807703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:38.807703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:38.807703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:38.808102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:38.808102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:38.808102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:38.808102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:38.808102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:38.808102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:38.808102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:38.808102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:38.808102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:38.808102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:38.808102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:38.808102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:38.808102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:38.808436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.808436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:38.808704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.808704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:38.808704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:38.808704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:38.808704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:38.808704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:38.808704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.808704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:38.808704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:38.808704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.808704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:38.808704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:38.808704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.808704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:38.808704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:38.808704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:38.809139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:15:38.809139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:38.809139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:38.809139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:38.809139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:38.809139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:38.809139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:38.809139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:38.809139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:38.809139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:38.809139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:38.809139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:38.809139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:38.809139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:38.809139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:38.809139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:38.809139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:38.809139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:38.809139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:38.809646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:38.809646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:38.809646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:42.965533 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:42.966486 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:42.966727 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:42.968004 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:43.002606 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:43.003147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.004404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:43.004404 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:43.004404 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:43.004404 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:43.004404 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:43.004404 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:43.004404 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.004404 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:43.005693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.005693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:43.005693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:43.005693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:43.005693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.005693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:15:43.005693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:43.005693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:43.005693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:43.005693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:43.005693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:43.005693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:43.006296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:43.006296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:43.006296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:43.006296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:43.006296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:43.006296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:43.006296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:43.006296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:43.006296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:43.006296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:43.006296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:43.006296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:43.006296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:43.006833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.006833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:43.007214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.007214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:43.007214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:43.007214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:43.007214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:43.007214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:43.007214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.007214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:43.007214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:43.007214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.007214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:43.007214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:43.007214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.007214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:43.007214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:43.007214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.007706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:15:43.007706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:43.007706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:43.007706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:43.007706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:43.007706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:43.007706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:43.007706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:43.007706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:43.007706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:43.007706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:43.007706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:43.007706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:43.007706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:43.007706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:43.007706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:43.007706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:43.007706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:43.007706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:43.008240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:43.008240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.008240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:43.741929 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:43.742525 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:43.742839 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:43.743695 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:43.743990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.744880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:43.744880 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:43.744880 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:43.744880 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:43.744880 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:43.744880 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:43.744880 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.744880 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:43.746348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.746348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:43.746348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:43.746348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:43.746348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.746348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:15:43.746348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:43.746348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:43.746348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:43.746348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:43.746348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:43.746348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:43.747087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:43.747087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:43.747087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:43.747087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:43.747087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:43.747087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:43.747087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:43.747087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:43.747087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:43.747087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:43.747087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:43.747087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:43.747087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:43.747648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.747648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:43.748212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.748212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:43.748212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:43.748212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:43.748212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:43.748212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:43.748212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.748212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:43.748212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:43.748212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.748212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:43.748212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:43.748212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.748212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:43.748212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:43.748212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.749043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:15:43.749043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:43.749043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:43.749043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:43.749043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:43.749043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:43.749043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:43.749043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:43.749043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:43.749043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:43.749043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:43.749043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:43.749043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:43.749043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:43.749043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:43.749043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:43.749043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:43.749043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:43.749043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:43.750506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:43.750506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.750506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:43.791056 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:43.791632 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:43.792005 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:43.792088 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:15:43.792601 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:43.793013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.793884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:43.793884 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:43.793884 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:43.793884 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:43.793884 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:43.793884 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:43.793884 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.793884 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:43.795219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.795219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:15:43.795219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:43.795219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:43.795219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:43.795219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:43.795219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:43.795219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:43.795219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:43.795219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:43.795219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:43.795219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:15:43.795219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:43.795219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:15:43.795219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:15:43.795652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:43.795652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:15:43.795652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:43.795652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:15:43.795652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:15:43.795652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:43.795652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:15:43.795652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.795652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:15:43.796651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.796651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:43.796651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:43.796651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:43.796651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.796651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:15:43.796651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:15:43.796651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:15:43.796651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:43.796651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:43.796651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:43.796651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:43.797025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:43.797025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:43.797025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:43.797025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:43.797025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:43.797025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:43.797025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:43.797025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:43.797025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:43.797025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:43.797025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:43.797025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:43.797025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:43.797025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:43.797426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:43.797426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:43.797426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.797426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:43.799402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.799402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:43.799402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:43.799402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:43.799402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:43.799402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:43.799402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.799402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:43.799402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:43.799402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.799402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:43.799402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:43.799402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.799402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:43.799402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:43.799402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.799906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:15:43.799906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:15:43.799906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:15:43.799906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:43.799906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:43.799906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:43.799906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:43.799906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:43.799906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:43.799906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:43.799906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:43.799906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:43.799906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:43.799906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:43.799906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:43.799906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:43.799906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:43.799906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:43.800446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:43.800446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:43.800446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:43.800446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:43.800446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:43.800446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.800446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:43.800655 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:43.800931 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:43.801121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:43.801522 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:43.801843 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:43.802225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.803031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:43.803031 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:43.803031 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:43.803031 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:43.803031 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:43.803031 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:43.803031 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.803031 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:43.804687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.804687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:15:43.804687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:43.804687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:43.804687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:43.804687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:43.804687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:43.804687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:43.804687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:43.804687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:43.804687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:43.804687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:15:43.804687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:43.804687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:15:43.804687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:15:43.805156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:43.805156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:15:43.805156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:43.805156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:15:43.805156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:15:43.805156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:43.805156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:15:43.805156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.805156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:15:43.805623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.805623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:43.805623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:43.805623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:43.805623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.805623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:15:43.805623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:15:43.805623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:43.805623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:43.805623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:43.805623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:43.805978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:43.805978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:43.805978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:43.805978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:43.805978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:43.805978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:43.805978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:43.805978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:43.805978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:43.805978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:43.805978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:43.805978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:43.805978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:43.805978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:43.806363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:43.806363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:43.806363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.806363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:43.807922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.807922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:43.807922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:43.807922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:43.807922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:43.807922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:43.807922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.807922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:43.807922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:43.807922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.807922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:43.807922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:43.807922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.807922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:43.807922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:43.807922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.807922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:15:43.808494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:15:43.808494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:43.808494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:43.808494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:43.808494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:43.808494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:43.808494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:43.808494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:43.808494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:43.808494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:43.808494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:43.808494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:43.808494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:43.808494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:43.808494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:43.808494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:43.808494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:43.808494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:43.808494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:43.809145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:43.809145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:43.809145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.809145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:43.809145 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:43.809145 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:43.809145 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:43.809427 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:43.809594 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:43.809745 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:43.809940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.810210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:43.810210 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:43.810210 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:43.810210 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:43.810210 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:43.810210 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:43.810210 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.810210 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:43.810877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.810877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:43.810877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:43.810877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:43.810877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.810877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:15:43.810877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:43.810877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:43.810877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:43.810877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:43.810877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:43.810877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:43.811294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:43.811294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:43.811294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:43.811294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:43.811294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:43.811294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:43.811294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:43.811294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:43.811294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:43.811294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:43.811294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:43.811294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:43.811294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:43.811653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.811653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:43.811653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.811653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:43.811653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:43.811653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:43.811653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:43.811653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:43.811653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.811653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:43.811653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:43.811653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.811653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:43.811653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:43.811653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.812138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:43.812138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:43.812138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:43.812138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:15:43.812138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:43.812138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:43.812138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:43.812138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:43.812138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:43.812138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:43.812138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:43.812138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:43.812138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:43.812138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:43.812138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:43.812138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:43.812138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:43.812138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:43.812138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:43.812766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:43.812766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:43.812766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:43.812766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:43.812766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:43.812766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:47.966362 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:47.967006 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:47.967120 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:47.968428 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:47.997064 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:47.997465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:47.998320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:47.998320 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:47.998320 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:47.998320 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:47.998320 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:47.998320 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:47.998320 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:47.998320 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:47.999118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:47.999118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:47.999118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:47.999118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:47.999118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:47.999118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:15:47.999118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:47.999118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:47.999118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:47.999118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:47.999118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:47.999118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:47.999642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:47.999642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:47.999642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:47.999642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:47.999642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:47.999642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:47.999642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:47.999642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:47.999642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:47.999642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:47.999642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:47.999642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:47.999642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:48.000297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.000297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:48.000297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.000297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:48.000297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:48.000297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:48.000297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:48.000297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:48.000297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:48.000297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:48.000297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:48.000297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:48.000297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:48.000297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:48.000297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:48.000951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:48.000951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:48.000951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:48.000951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:15:48.000951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:48.000951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:48.000951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:48.000951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:48.000951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:48.000951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:48.000951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:48.000951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:48.000951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:48.000951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:48.000951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:48.000951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:48.000951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:48.000951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:48.000951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:48.004032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:48.004032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:48.004032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:48.004032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:48.004032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.004032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:48.742619 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:48.743278 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:48.743464 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:48.744331 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:48.745170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.745457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:48.745457 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:48.745457 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:48.745457 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:48.745457 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:48.745457 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:48.745457 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.745457 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:48.746763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.746763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:48.746763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:48.746763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:48.746763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:48.746763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:15:48.746763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:48.746763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:48.746763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:48.746763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:48.746763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:48.746763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:48.747407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:48.747407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:48.747407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:48.747407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:48.747407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:48.747407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:48.747407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:48.747407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:48.747407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:48.747407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:48.747407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:48.747407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:48.747407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:48.748092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.748092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:48.748092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.748092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:48.748092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:48.748092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:48.748092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:48.748092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:48.748092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:48.748092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:48.748092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:48.748092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:48.748092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:48.748092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:48.748092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:48.748915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:48.748915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:48.748915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:48.748915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:15:48.748915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:48.748915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:48.748915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:48.748915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:48.748915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:48.748915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:48.748915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:48.748915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:48.748915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:48.748915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:48.748915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:48.748915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:48.748915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:48.748915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:48.748915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:48.750092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:48.750092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:48.750092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:48.750092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:48.750092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.750092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:48.792312 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:48.793853 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:48.793853 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:48.793853 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:15:48.794669 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:48.795018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.795579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:48.795579 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:48.795579 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:48.795579 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:48.795579 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:48.795579 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:48.795579 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.795579 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:48.796880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.796880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:15:48.796880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:48.796880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:48.796880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:48.796880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:48.796880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:48.796880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:48.796880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:48.796880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:48.796880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:48.796880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:15:48.796880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:48.796880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:15:48.796880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:15:48.797796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:48.797796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:15:48.797796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:48.797796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:15:48.797796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:15:48.797796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:48.797796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:15:48.797796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.797796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:15:48.798243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.798243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:48.798243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:48.798243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:48.798243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:48.798243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:15:48.798243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:15:48.798243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:15:48.798243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:48.798243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:48.798243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:48.798243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:48.798768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:48.798768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:48.798768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:48.798768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:48.798768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:48.798768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:48.798768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:48.798768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:48.798768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:48.798768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:48.798768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:48.798768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:48.798768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:48.798768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:48.799360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:48.799360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:48.799360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.799360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:48.799360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.799360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:48.799360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:48.799360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:48.799360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:48.799360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:48.799360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:48.799360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:48.799360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:48.799360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:48.800014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:48.800014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:48.800014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:48.800014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:48.800014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:48.800014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:48.800014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:15:48.800014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:15:48.800014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:15:48.800014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:48.800014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:48.800014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:48.800014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:48.800014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:48.800014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:48.800014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:48.800014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:48.800014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:48.800014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:48.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:48.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:48.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:48.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:48.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:48.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:48.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:48.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:48.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:48.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:48.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:48.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:48.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:48.801872 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:48.801872 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:48.801872 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:48.801872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.802241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:48.802241 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:48.802241 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:48.802241 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:48.802241 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:48.802241 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:48.802241 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.802241 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:48.802780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.802780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:15:48.802780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:48.802780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:48.802780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:48.802780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:48.802780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:48.802780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:48.802780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:48.802780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:48.802780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:48.802780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:15:48.802780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:48.802780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:15:48.802780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:15:48.803183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:48.803183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:15:48.803183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:48.803183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:15:48.803183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:15:48.803183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:48.803183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:15:48.803183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.803183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:15:48.803434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.803434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:48.803434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:48.803434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:48.803434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:48.803434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:15:48.803434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:15:48.803434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:48.803434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:48.803434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:48.803434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:48.803747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:48.803747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:48.803747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:48.803747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:48.803747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:48.803747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:48.803747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:48.803747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:48.803747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:48.803747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:48.803747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:48.803747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:48.803747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:48.803747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:48.804125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:48.804125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:48.804125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.804125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:48.804125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.804125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:48.804125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:48.804125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:48.804125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:48.804125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:48.804125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:48.804125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:48.804125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:48.804125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:48.804504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:48.804504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:48.804504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:48.804504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:48.804504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:48.804504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:48.804504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:15:48.804504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:15:48.804504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:48.804504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:48.804504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:48.804504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:48.804504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:48.804504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:48.804504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:48.804504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:48.804504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:48.804504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:48.804504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:48.805456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:48.805456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:48.805456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:48.805456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:48.805456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:48.805456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:48.805456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:48.805456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:48.805456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:48.805456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.805456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:48.808792 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:48.809040 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:48.809154 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:48.809323 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:48.809652 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:48.810134 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:48.810320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.811293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:48.811293 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:48.811293 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:48.811293 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:48.811293 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:48.811293 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:48.811293 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.811293 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:48.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:48.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:48.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:48.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:48.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:15:48.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:48.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:48.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:48.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:48.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:48.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:48.812494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:48.812494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:48.812494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:48.812494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:48.812494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:48.812494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:48.812494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:48.812494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:48.812494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:48.812494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:48.812494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:48.812494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:48.812494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:48.812870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.812870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:48.813070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.813070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:48.813070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:48.813070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:48.813070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:48.813070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:48.813070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:48.813070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:48.813070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:48.813070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:48.813070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:48.813070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:48.813070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:48.813070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:48.813070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:48.813070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:48.813720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:15:48.813720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:48.813720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:48.813720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:48.813720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:48.813720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:48.813720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:48.813720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:48.813720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:48.813720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:48.813720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:48.813720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:48.813720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:48.813720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:48.813720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:48.813720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:48.813720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:48.813720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:48.813720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:48.814185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:48.814185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:48.814185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:52.727007 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=719abecd-2f6c-40a7-ad6d-e00b9e819e78, enabled=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-8a1fb774-5693-4139-8b75-6ab5273228c7', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d', 'neutron:subnet_ids': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece 335d12ce-2d0c-4086-b3f4-2af390c03d70'}, gateway_chassis=[], ha_chassis_group=68a12f6b-12f9-4d1e-a747-50d853aa8ef8, ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:f8:07:38', name='lrp-73493630-b0ac-48ec-be98-5b048d2e2ce7', networks=['172.24.5.87/24', '2001:db8::2d8/64'], options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}, peer=[], status={}) old= {{(pid=106506) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
Apr 28 20:15:52.967089 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:52.967789 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:52.968031 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:52.969695 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:53.004399 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:53.004852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.005648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:53.005648 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:53.005648 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:53.005648 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:53.005648 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:53.005648 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:53.005648 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.005648 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:53.006696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.006696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:53.006696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:53.006696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:53.006696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.006696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:15:53.006696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:53.006696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:53.006696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:53.006696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:53.006696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:53.006696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:53.007309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:53.007309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:53.007309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:53.007309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:53.007309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:53.007309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:53.007309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:53.007309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:53.007309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:53.007309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:53.007309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:53.007309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:53.007309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:53.007960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.007960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:53.007960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.007960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:53.007960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:53.007960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:53.007960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:53.007960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:53.007960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.007960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:53.007960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:53.007960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.007960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:53.007960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:53.007960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.008611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:53.008611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:53.008611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.008611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:15:53.008611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:53.008611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:53.008611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:53.008611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:53.008611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:53.008611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:53.008611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:53.008611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:53.008611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:53.008611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:53.008611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:53.008611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:53.008611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:53.008611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:53.008611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:53.009550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:53.009550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:53.009550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:53.009550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:53.009550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.009550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:53.743385 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:53.744200 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:53.744342 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:53.745451 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:53.745844 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:53.746067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.746956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:53.746956 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:53.746956 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:53.746956 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:53.746956 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:53.746956 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:53.746956 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.746956 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:53.748800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.748800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:53.748800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:53.748800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:53.748800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.748800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:15:53.748800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:53.748800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:53.748800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:53.748800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:53.748800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:53.748800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:53.749345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:53.749345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:53.749345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:53.749345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:53.749345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:53.749345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:53.749345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:53.749345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:53.749345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:53.749345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:53.749345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:53.749345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:53.749345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:53.750119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.750119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:53.750533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.750533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:53.750533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:53.750533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:53.750533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:53.750533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:53.750533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.750533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:53.750533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:53.750533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.750533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:53.750533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:53.750533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.750533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:53.750533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:53.750533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.751248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:15:53.751248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:53.751248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:53.751248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:53.751248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:53.751248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:53.751248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:53.751248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:53.751248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:53.751248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:53.751248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:53.751248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:53.751248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:53.751248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:53.751248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:53.751248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:53.751248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:53.751248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:53.751248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:53.752095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:53.752095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.752095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:53.792719 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:53.793157 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:53.793381 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:53.793579 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:15:53.795112 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:53.795481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.796578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:53.796578 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:53.796578 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:53.796578 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:53.796578 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:53.796578 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:53.796578 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.796578 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:53.798019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.798019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:15:53.798019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:53.798019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:53.798019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:53.798019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:53.798019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:53.798019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:53.798019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:53.798019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:53.798019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:53.798019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:15:53.798019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:53.798019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:15:53.798019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:15:53.798670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:53.798670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:15:53.798670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:53.798670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:15:53.798670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:15:53.798670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:53.798670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:15:53.798670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.798670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:15:53.799383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.799383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:53.799383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:53.799383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:53.799383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.799383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:15:53.799383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:15:53.799383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:15:53.799383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:53.799383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:53.799383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:53.799383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:53.799959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:53.799959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:53.799959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:53.799959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:53.799959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:53.799959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:53.799959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:53.799959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:53.799959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:53.799959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:53.799959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:53.799959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:53.799959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:53.799959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:53.800551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:53.800551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:53.800551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.800551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:53.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:53.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:53.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:53.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:53.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:53.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:53.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:53.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:53.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:53.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:53.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:53.801070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.801789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:15:53.801789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:15:53.801789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:15:53.801789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:53.801789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:53.801789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:53.801789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:53.801789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:53.801789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:53.801789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:53.801789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:53.801789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:53.801789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:53.801789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:53.801789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:53.801789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:53.801789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:53.801789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:53.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:53.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:53.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:53.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:53.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:53.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:53.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:53.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:53.802559 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:53.803275 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:53.803275 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:53.803275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.803275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:53.803275 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:53.803275 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:53.803275 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:53.803275 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:53.803275 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:53.803275 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.803275 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:53.803960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.803960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:15:53.803960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:53.803960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:53.803960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:53.803960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:53.803960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:53.803960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:53.803960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:53.803960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:53.803960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:53.803960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:15:53.803960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:53.803960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:15:53.803960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:15:53.804615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:53.804615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:15:53.804615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:53.804615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:15:53.804615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:15:53.804615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:53.804615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:15:53.804615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.804615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:15:53.805035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.805035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:53.805035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:53.805035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:53.805035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.805035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:15:53.805035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:15:53.805035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:53.805035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:53.805035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:53.805035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:53.805591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:53.805591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:53.805591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:53.805591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:53.805591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:53.805591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:53.805591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:53.805591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:53.805591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:53.805591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:53.805591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:53.805591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:53.805591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:53.805591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:53.806233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:53.806233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:53.806233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.806233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:53.806233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.806233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:53.806233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:53.806233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:53.806233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:53.806233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:53.806233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.806233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:53.806233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:53.806233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.806852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:53.806852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:53.806852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.806852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:53.806852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:53.806852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.806852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:15:53.806852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:15:53.806852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:53.806852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:53.806852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:53.806852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:53.806852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:53.806852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:53.806852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:53.806852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:53.806852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:53.806852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:53.806852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:53.807651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:53.807651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:53.807651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:53.807651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:53.807651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:53.807651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:53.807651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:53.807651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:53.807651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:53.807651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.807651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:53.809196 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:53.809534 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:53.809654 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:53.809829 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:53.810302 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:53.810685 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:53.810987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.811740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:53.811740 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:53.811740 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:53.811740 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:53.811740 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:53.811740 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:53.811740 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.811740 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:53.812692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.812692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:53.812692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:53.812692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:53.812692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.812692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:15:53.812692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:53.812692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:53.812692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:53.812692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:53.812692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:53.812692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:53.812941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:53.812941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:53.812941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:53.812941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:53.812941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:53.812941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:53.812941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:53.812941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:53.812941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:53.812941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:53.812941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:53.812941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:53.812941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:53.813192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.813192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:53.813829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.813829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:53.813829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:53.813829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:53.813829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:53.813829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:53.813829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.813829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:53.813829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:53.813829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.813829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:53.813829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:53.813829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.813829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:53.813829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:53.813829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:53.814144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:15:53.814144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:53.814144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:53.814144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:53.814144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:53.814144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:53.814144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:53.814144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:53.814144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:53.814144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:53.814144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:53.814144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:53.814144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:53.814144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:53.814144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:53.814144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:53.814144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:53.814144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:53.814144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:53.814488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:53.814488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:53.814488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:56.094308 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(uuid=44adbf36-e130-46fd-afc8-8b25e7c6ad86, enabled=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-d3dfebbd-7943-4813-b509-739084ada63b', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d', 'neutron:subnet_ids': 'c68bc19b-c88a-4cd2-bf3a-d847d267d23f'}, gateway_chassis=[], ha_chassis_group=[], ipv6_prefix=[], ipv6_ra_configs={}, mac='fa:16:3e:e5:60:9f', name='lrp-409c09f7-cae9-446e-a7c3-0215740f677d', networks=['192.168.24.1/24'], options={}, peer=[], status={}) old= {{(pid=106506) matches /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}}
Apr 28 20:15:56.129603 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-d3dfebbd-7943-4813-b509-739084ada63b synchronized; highest priority chassis d6012b70-0a69-43bb-a268-b8c8213be6d4
Apr 28 20:15:56.130121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-d3dfebbd-7943-4813-b509-739084ada63b, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': 'd3dfebbd-7943-4813-b509-739084ada63b', 'neutron:router_id': '6feb526a-13c6-41ac-983d-ea1d2251ff3d'}}) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:56.130374 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-d3dfebbd-7943-4813-b509-739084ada63b, chassis=d6012b70-0a69-43bb-a268-b8c8213be6d4, priority=1, columns={}) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:56.130601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:56.131173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] Traceback (most recent call last):
Apr 28 20:15:56.131173 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:56.131173 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:56.131173 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:56.131173 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:56.131173 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:56.131173 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:56.131173 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:56.132145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event [-] Unexpected exception in notify_loop: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:56.132145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event Traceback (most recent call last):
Apr 28 20:15:56.132145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/event.py", line 185, in notify_loop
Apr 28 20:15:56.132145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     match.run(event, row, updates)
Apr 28 20:15:56.132145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 86, in run
Apr 28 20:15:56.132145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     self.l3_plugin._ovn_client.link_network_ha_chassis_group(
Apr 28 20:15:56.132145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2411, in link_network_ha_chassis_group
Apr 28 20:15:56.132145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:56.132145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:56.132145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     next(self.gen)
Apr 28 20:15:56.132145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:56.132145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:56.132145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:56.132145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     next(self.gen)
Apr 28 20:15:56.132145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:56.132145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     with self.create_transaction(
Apr 28 20:15:56.132145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:56.132145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     self.result = self.commit()
Apr 28 20:15:56.132145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event                   ^^^^^^^^^^^^^
Apr 28 20:15:56.133428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:56.133428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     raise result.ex
Apr 28 20:15:56.133428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:56.133428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     txn.results.put(txn.do_commit())
Apr 28 20:15:56.133428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event                     ^^^^^^^^^^^^^^^
Apr 28 20:15:56.133428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:56.133428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event     raise RuntimeError(msg)
Apr 28 20:15:56.133428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:56.133428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.event 
Apr 28 20:15:57.967744 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:57.968428 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:57.968696 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:57.970386 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:58.003698 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:58.004125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.005043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:58.005043 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:58.005043 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:58.005043 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:58.005043 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:58.005043 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:58.005043 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.005043 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:58.005964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.005964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:58.005964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:58.005964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:58.005964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.005964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:15:58.005964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:58.005964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:58.005964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:58.005964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:58.005964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:58.005964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:58.006430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:58.006430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:58.006430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:58.006430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:58.006430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:58.006430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:58.006430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:58.006430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:58.006430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:58.006430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:58.006430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:58.006430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:58.006430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:58.006899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.006899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:58.007102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.007102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:58.007102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:58.007102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:58.007102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:58.007102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:58.007102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.007102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:58.007102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:58.007102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.007102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:58.007102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:58.007102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.007102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:58.007102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:58.007102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.007512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:15:58.007512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:58.007512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:58.007512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:58.007512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:58.007512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:58.007512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:58.007512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:58.007512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:58.007512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:58.007512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:58.007512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:58.007512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:58.007512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:58.007512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:58.007512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:58.007512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:58.007512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:58.007512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:58.007902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:58.007902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.007902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:58.744088 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:58.744560 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:58.744824 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:58.746050 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:58.746435 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:58.746792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.747809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:58.747809 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:58.747809 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:58.747809 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:58.747809 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:58.747809 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:58.747809 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.747809 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:58.749475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.749475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:58.749475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:58.749475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:58.749475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.749475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:15:58.749475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:58.749475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:58.749475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:58.749475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:58.749475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:58.749475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:58.750081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:58.750081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:58.750081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:58.750081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:58.750081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:58.750081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:58.750081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:58.750081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:58.750081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:58.750081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:58.750081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:58.750081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:58.750081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:58.750720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.750720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:58.751302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.751302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:58.751302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:58.751302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:58.751302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:58.751302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:58.751302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.751302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:58.751302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:58.751302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.751302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:58.751302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:58.751302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.751302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:58.751302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:58.751302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.752023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:15:58.752023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:58.752023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:58.752023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:58.752023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:58.752023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:58.752023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:58.752023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:58.752023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:58.752023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:58.752023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:58.752023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:58.752023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:58.752023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:58.752023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:58.752023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:58.752023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:58.752023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:58.752023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:58.752813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:58.752813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.752813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:58.793204 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:58.793602 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:58.793934 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:58.794001 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:15:58.798361 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:58.798756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.799486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:58.799486 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:58.799486 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:58.799486 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:58.799486 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:58.799486 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:58.799486 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.799486 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:58.800665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.800665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:15:58.800665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:58.800665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:58.800665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:58.800665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:58.800665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:58.800665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:58.800665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:58.800665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:58.800665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:58.800665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:15:58.800665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:58.800665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:15:58.800665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:15:58.801241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:58.801241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:15:58.801241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:58.801241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:15:58.801241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:15:58.801241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:58.801241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:15:58.801241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.801241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:15:58.801984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.801984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:58.801984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:58.801984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:58.801984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.801984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:15:58.801984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:15:58.801984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:15:58.801984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:58.801984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:58.801984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:58.801984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:58.802357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:58.802357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:58.802357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:58.802357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:58.802357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:58.802357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:58.802357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:58.802357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:58.802357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:58.802357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:58.802357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:58.802357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:58.802357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:58.802357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:58.802786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:58.802786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:58.802786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.802786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:58.804281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.804281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:58.804281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:58.804281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:58.804281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:58.804281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:58.804281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.804281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:58.804281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:58.804281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.804281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:58.804281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:58.804281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.804281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:58.804281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:58.804281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.804773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:15:58.804773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:15:58.804773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:15:58.804773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:58.804773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:58.804773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:58.804773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:58.804773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:58.804773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:58.804773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:58.804773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:58.804773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:58.804773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:58.804773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:58.804773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:58.804773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:58.804773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:58.804773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:58.805277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:58.805277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:58.805277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:58.805277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:58.805277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:58.805277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.805277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:58.805277 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:58.805277 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:58.805277 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:58.805591 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:58.806008 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:58.806350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.807206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:58.807206 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:58.807206 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:58.807206 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:58.807206 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:58.807206 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:58.807206 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.807206 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:58.808440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.808440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:15:58.808440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:58.808440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:58.808440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:58.808440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:58.808440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:58.808440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:58.808440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:58.808440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:15:58.808440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:58.808440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:15:58.808440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:58.808440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:15:58.808440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:15:58.808894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:58.808894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:15:58.808894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:58.808894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:15:58.808894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:15:58.808894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:58.808894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:15:58.808894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.808894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:15:58.810055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.810055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:58.810055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:58.810055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:58.810055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.810055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:15:58.810055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:15:58.810055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:58.810055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:58.810055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:58.810055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:58.810403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:58.810403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:58.810403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:58.810403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:58.810403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:58.810403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:58.810403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:58.810403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:58.810403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:58.810403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:58.810403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:58.810403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:58.810403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:58.810403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:58.810839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:58.810839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:58.810839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.810839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:58.811689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.811689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:58.811689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:58.811689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:58.811689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:58.811689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:58.811689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.811689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:58.811689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:58.811689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.811689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:58.811689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:58.811689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.811689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:58.811689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:58.811689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.811689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:15:58.812195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:15:58.812195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:15:58.812195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:15:58.812195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:58.812195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:58.812195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:58.812195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:58.812195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:58.812195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:58.812195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:58.812195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:58.812195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:58.812195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:58.812195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:58.812195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:58.812195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:58.812195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:58.812195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:58.812195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:58.812743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:58.812743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:58.812743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.812743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:15:58.812743 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:15:58.812743 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:15:58.812743 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:15:58.812743 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:15:58.813017 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:58.813298 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:15:58.813455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.813997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:15:58.813997 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:58.813997 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:15:58.813997 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:15:58.813997 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:58.813997 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:15:58.813997 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.813997 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:15:58.814810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.814810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:15:58.814810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:58.814810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:15:58.814810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.814810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:15:58.814810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:58.814810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:58.814810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:58.814810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:58.814810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:58.814810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:58.815393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:15:58.815393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:58.815393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:15:58.815393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:58.815393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:15:58.815393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:15:58.815393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:58.815393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:15:58.815393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:58.815393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:15:58.815393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:15:58.815393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:58.815393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:15:58.815769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.815769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:15:58.815769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.815769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:15:58.815769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:15:58.815769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:15:58.815769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:15:58.815769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:15:58.815769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.815769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:15:58.815769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:15:58.815769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.815769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:15:58.815769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:15:58.815769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.816235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:15:58.816235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:15:58.816235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:15:58.816235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:15:58.816235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:15:58.816235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:58.816235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:58.816235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:15:58.816235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:15:58.816235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:15:58.816235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:15:58.816235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:15:58.816235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:15:58.816235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:15:58.816235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:15:58.816235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:15:58.816235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:15:58.816235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:15:58.816235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:15:58.816786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:15:58.816786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:15:58.816786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:15:58.816786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:15:58.816786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:15:58.816786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:02.968386 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:02.969342 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:02.969627 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:02.971224 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:02.979828 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=250 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:16:03.011559 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:03.011766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.012504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:03.012504 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:03.012504 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:03.012504 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:03.012504 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:03.012504 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:03.012504 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.012504 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:03.014135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.014135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:03.014135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:03.014135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:03.014135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.014135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:16:03.014135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:03.014135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:03.014135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:03.014135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:03.014135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:03.014135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:03.015114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:03.015114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:03.015114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:03.015114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:03.015114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:03.015114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:03.015114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:03.015114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:03.015114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:03.015114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:03.015114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:03.015114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:03.015114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:03.015786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.015786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:03.015786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.015786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:03.015786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:03.015786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:03.015786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:03.015786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:03.015786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.015786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:03.015786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:03.015786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.015786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:03.015786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:03.015786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.016458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:03.016458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:03.016458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.016458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:16:03.016458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:03.016458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:03.016458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:03.016458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:03.016458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:03.016458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:03.016458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:03.016458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:03.016458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:03.016458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:03.016458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:03.016458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:03.016458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:03.016458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:03.016458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:03.017288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:03.017288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:03.017288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:03.017288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:03.017288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.017288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:03.744560 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:03.745244 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:03.745432 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:03.746549 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:03.747012 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:03.747391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.748427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:03.748427 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:03.748427 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:03.748427 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:03.748427 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:03.748427 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:03.748427 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.748427 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:03.750383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.750383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:03.750383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:03.750383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:03.750383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.750383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:16:03.750383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:03.750383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:03.750383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:03.750383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:03.750383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:03.750383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:03.751462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:03.751462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:03.751462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:03.751462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:03.751462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:03.751462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:03.751462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:03.751462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:03.751462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:03.751462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:03.751462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:03.751462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:03.751462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:03.752436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.752436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:03.752819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.752819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:03.752819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:03.752819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:03.752819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:03.752819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:03.752819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.752819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:03.752819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:03.752819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.752819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:03.752819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:03.752819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.752819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:03.752819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:03.752819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.753287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:16:03.753287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:03.753287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:03.753287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:03.753287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:03.753287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:03.753287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:03.753287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:03.753287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:03.753287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:03.753287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:03.753287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:03.753287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:03.753287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:03.753287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:03.753287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:03.753287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:03.753287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:03.753287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:03.753768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:03.753768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.753768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:03.793700 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:03.794128 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:03.794298 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:03.794481 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:16:03.795203 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:03.795704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.796409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:03.796409 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:03.796409 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:03.796409 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:03.796409 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:03.796409 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:03.796409 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.796409 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:03.797891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.797891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:16:03.797891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:03.797891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:03.797891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:03.797891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:03.797891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:03.797891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:03.797891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:03.797891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:03.797891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:03.797891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:16:03.797891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:03.797891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:16:03.797891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:16:03.798862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:03.798862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:16:03.798862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:03.798862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:16:03.798862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:16:03.798862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:03.798862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:16:03.798862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.798862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:16:03.799301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.799301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:03.799301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:03.799301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:03.799301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.799301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:16:03.799301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:16:03.799301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:16:03.799301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:03.799301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:03.799301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:03.799301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:03.799854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:03.799854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:03.799854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:03.799854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:03.799854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:03.799854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:03.799854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:03.799854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:03.799854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:03.799854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:03.799854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:03.799854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:03.799854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:03.799854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:03.800474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:03.800474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:03.800474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.800474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:03.800474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.800474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:03.800474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:03.800474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:03.800474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:03.800474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:03.800474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.800474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:03.800474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:03.800474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.801138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:03.801138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:03.801138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.801138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:03.801138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:03.801138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.801138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:16:03.801138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:16:03.801138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:16:03.801138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:03.801138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:03.801138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:03.801138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:03.801138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:03.801138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:03.801138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:03.801138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:03.801138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:03.801138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:03.804140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:03.804140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:03.804140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:03.804140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:03.804140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:03.804140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:03.804140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:03.804140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:03.804140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:03.804140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:03.804140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.804140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:03.804771 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:03.805216 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:03.805216 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:03.805318 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:03.805615 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:03.806067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.806432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:03.806432 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:03.806432 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:03.806432 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:03.806432 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:03.806432 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:03.806432 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.806432 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:03.807391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.807391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:16:03.807391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:03.807391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:03.807391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:03.807391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:03.807391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:03.807391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:03.807391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:03.807391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:03.807391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:03.807391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:16:03.807391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:03.807391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:16:03.807391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:16:03.808108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:03.808108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:16:03.808108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:03.808108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:16:03.808108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:16:03.808108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:03.808108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:16:03.808108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.808108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:16:03.808899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.808899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:03.808899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:03.808899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:03.808899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.808899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:16:03.808899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:16:03.808899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:03.808899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:03.808899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:03.808899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:03.809470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:03.809470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:03.809470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:03.809470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:03.809470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:03.809470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:03.809470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:03.809470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:03.809470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:03.809470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:03.809470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:03.809470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:03.809470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:03.809470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:03.810003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:03.810003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:03.810003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.810003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:03.810787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.810787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:03.810787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:03.810787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:03.810787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:03.810787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:03.810787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.810787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:03.810787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:03.810787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.810787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:03.810787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:03.810787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.810787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:03.810787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:03.810787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.810787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:16:03.811703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:16:03.811703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:03.811703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:03.811703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:03.811703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:03.811703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:03.811703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:03.811703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:03.811703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:03.811703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:03.811703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:03.811703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:03.811703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:03.811703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:03.811703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:03.811703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:03.811703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:03.811703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:03.811703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:03.812700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:03.812700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:03.812700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.812700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:03.812700 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:03.812700 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:03.812700 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:03.812700 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:03.813089 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:03.813390 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:03.813607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.814141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:03.814141 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:03.814141 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:03.814141 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:03.814141 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:03.814141 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:03.814141 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.814141 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:03.815051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.815051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:03.815051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:03.815051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:03.815051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.815051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:16:03.815051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:03.815051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:03.815051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:03.815051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:03.815051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:03.815051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:03.815419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:03.815419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:03.815419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:03.815419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:03.815419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:03.815419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:03.815419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:03.815419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:03.815419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:03.815419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:03.815419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:03.815419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:03.815419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:03.815741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.815741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:03.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:03.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:03.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:03.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:03.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:03.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:03.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:03.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:03.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:03.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:03.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:03.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:03.816747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:16:03.816747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:03.816747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:03.816747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:03.816747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:03.816747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:03.816747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:03.816747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:03.816747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:03.816747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:03.816747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:03.816747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:03.816747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:03.816747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:03.816747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:03.816747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:03.816747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:03.816747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:03.816747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:03.817595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:03.817595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:03.817595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:07.969022 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:07.969837 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:07.970035 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:07.971491 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:08.008521 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:08.008668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.009236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:08.009236 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:08.009236 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:08.009236 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:08.009236 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:08.009236 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:08.009236 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.009236 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:08.009919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.009919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:08.009919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:08.009919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:08.009919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.009919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:16:08.009919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:08.009919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:08.009919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:08.009919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:08.009919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:08.009919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:08.010492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:08.010492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:08.010492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:08.010492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:08.010492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:08.010492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:08.010492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:08.010492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:08.010492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:08.010492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:08.010492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:08.010492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:08.010492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:08.011099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.011099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:08.011099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.011099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:08.011099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:08.011099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:08.011099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:08.011099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:08.011099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.011099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:08.011099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:08.011099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.011099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:08.011099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:08.011099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.011851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:08.011851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:08.011851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.011851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:16:08.011851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:08.011851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:08.011851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:08.011851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:08.011851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:08.011851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:08.011851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:08.011851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:08.011851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:08.011851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:08.011851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:08.011851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:08.011851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:08.011851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:08.011851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:08.012654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:08.012654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:08.012654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:08.012654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:08.012654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.012654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:08.745202 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:08.745780 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:08.745874 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:08.746830 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:08.747027 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:08.747182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.747933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:08.747933 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:08.747933 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:08.747933 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:08.747933 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:08.747933 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:08.747933 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.747933 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:08.748884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.748884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:08.748884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:08.748884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:08.748884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.748884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:16:08.748884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:08.748884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:08.748884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:08.748884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:08.748884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:08.748884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:08.749561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:08.749561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:08.749561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:08.749561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:08.749561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:08.749561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:08.749561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:08.749561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:08.749561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:08.749561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:08.749561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:08.749561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:08.749561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:08.750159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.750159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:08.750159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.750159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:08.750159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:08.750159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:08.750159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:08.750159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:08.750159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.750159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:08.750159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:08.750159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.750159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:08.750159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:08.750159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.750831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:08.750831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:08.750831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.750831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:16:08.750831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:08.750831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:08.750831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:08.750831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:08.750831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:08.750831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:08.750831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:08.750831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:08.750831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:08.750831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:08.750831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:08.750831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:08.750831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:08.750831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:08.750831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:08.751623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:08.751623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:08.751623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:08.751623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:08.751623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.751623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:08.794255 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:08.795041 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:08.795391 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:08.795592 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:16:08.796904 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:08.797131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.798190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:08.798190 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:08.798190 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:08.798190 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:08.798190 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:08.798190 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:08.798190 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.798190 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:08.799712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.799712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:16:08.799712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:08.799712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:08.799712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:08.799712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:08.799712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:08.799712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:08.799712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:08.799712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:08.799712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:08.799712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:16:08.799712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:08.799712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:16:08.799712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:16:08.800567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:08.800567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:16:08.800567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:08.800567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:16:08.800567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:16:08.800567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:08.800567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:16:08.800567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.800567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:16:08.800931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.800931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:08.800931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:08.800931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:08.800931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.800931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:16:08.800931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:16:08.800931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:16:08.800931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:08.800931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:08.800931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:08.800931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:08.801306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:08.801306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:08.801306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:08.801306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:08.801306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:08.801306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:08.801306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:08.801306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:08.801306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:08.801306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:08.801306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:08.801306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:08.801306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:08.801306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:08.801772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:08.801772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:08.801772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.801772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:08.802395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.802395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:08.802395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:08.802395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:08.802395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:08.802395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:08.802395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.802395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:08.802395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:08.802395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.802395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:08.802395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:08.802395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.802395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:08.802395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:08.802395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.802891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:16:08.802891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:16:08.802891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:16:08.802891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:08.802891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:08.802891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:08.802891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:08.802891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:08.802891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:08.802891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:08.802891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:08.802891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:08.802891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:08.802891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:08.802891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:08.802891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:08.802891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:08.802891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:08.803401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:08.803401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:08.803401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:08.803401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:08.803401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:08.803401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.803401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:08.805187 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:08.805578 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:08.805706 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:08.805923 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:08.806370 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:08.806723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.807361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:08.807361 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:08.807361 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:08.807361 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:08.807361 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:08.807361 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:08.807361 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.807361 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:08.808313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.808313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:16:08.808313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:08.808313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:08.808313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:08.808313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:08.808313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:08.808313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:08.808313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:08.808313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:08.808313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:08.808313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:16:08.808313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:08.808313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:16:08.808313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:16:08.808854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:08.808854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:16:08.808854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:08.808854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:16:08.808854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:16:08.808854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:08.808854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:16:08.808854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.808854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:16:08.809298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.809298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:08.809298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:08.809298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:08.809298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.809298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:16:08.809298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:16:08.809298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:08.809298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:08.809298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:08.809298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:08.809695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:08.809695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:08.809695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:08.809695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:08.809695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:08.809695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:08.809695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:08.809695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:08.809695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:08.809695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:08.809695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:08.809695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:08.809695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:08.809695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:08.810079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:08.810079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:08.810079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.810079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:08.810760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.810760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:08.810760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:08.810760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:08.810760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:08.810760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:08.810760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.810760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:08.810760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:08.810760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.810760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:08.810760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:08.810760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.810760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:08.810760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:08.810760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.810760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:16:08.811245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:16:08.811245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:08.811245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:08.811245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:08.811245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:08.811245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:08.811245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:08.811245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:08.811245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:08.811245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:08.811245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:08.811245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:08.811245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:08.811245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:08.811245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:08.811245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:08.811245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:08.811245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:08.811245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:08.811882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:08.811882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:08.811882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.811882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:08.812748 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:08.813147 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:08.813273 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:08.813465 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:08.813983 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:08.814354 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:08.814592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.815379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:08.815379 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:08.815379 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:08.815379 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:08.815379 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:08.815379 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:08.815379 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.815379 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:08.816433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.816433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:08.816433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:08.816433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:08.816433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.816433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:16:08.816433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:08.816433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:08.816433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:08.816433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:08.816433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:08.816433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:08.816897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:08.816897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:08.816897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:08.816897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:08.816897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:08.816897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:08.816897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:08.816897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:08.816897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:08.816897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:08.816897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:08.816897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:08.816897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:08.817267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.817267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:08.817590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.817590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:08.817590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:08.817590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:08.817590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:08.817590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:08.817590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.817590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:08.817590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:08.817590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.817590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:08.817590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:08.817590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.817590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:08.817590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:08.817590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:08.818156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:16:08.818156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:08.818156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:08.818156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:08.818156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:08.818156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:08.818156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:08.818156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:08.818156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:08.818156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:08.818156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:08.818156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:08.818156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:08.818156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:08.818156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:08.818156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:08.818156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:08.818156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:08.818156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:08.818854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:08.818854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:08.818854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:12.969864 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:12.970791 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:12.971002 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:12.972481 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:13.019581 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:13.019746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.020397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:13.020397 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:13.020397 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:13.020397 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:13.020397 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:13.020397 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:13.020397 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.020397 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:13.022087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.022087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:13.022087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:13.022087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:13.022087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.022087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:16:13.022087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:13.022087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.022087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:13.022087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:13.022087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:13.022087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.022875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:13.022875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:13.022875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:13.022875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:13.022875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:13.022875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:13.022875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:13.022875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:13.022875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:13.022875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:13.022875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:13.022875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:13.022875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:13.024857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.024857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:13.024857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.024857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:13.024857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:13.024857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:13.024857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:13.024857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:13.024857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.024857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:13.024857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:13.024857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.024857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:13.024857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:13.024857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:13.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:13.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:16:13.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:13.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:13.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:13.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:13.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:13.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:13.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:13.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:13.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:13.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:13.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:13.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:13.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:13.027773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:13.027773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:13.027773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:13.027773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:13.027773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.027773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:13.053510 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:13.053852 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:13.054067 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:13.062596 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
Apr 28 20:16:13.062874 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Number of inconsistencies found at create/update: networks=2, security_groups=1 {{(pid=106506) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
Apr 28 20:16:13.062971 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 7ff08466-af6d-4e13-bb1d-8987170a5734 (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:16:13.091965 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-7ff08466-af6d-4e13-bb1d-8987170a5734, resource={'id': '7ff08466-af6d-4e13-bb1d-8987170a5734', 'name': 'private', 'tenant_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'project_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['4b461bd9-09ea-4f87-a007-a11ab022ff7f', '8f3677a2-e2be-443a-a90c-2cd7b3095c49'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-28T20:10:58Z', 'updated_at': '2026-04-28T20:11:02Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 46342}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:13.092420 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-7ff08466-af6d-4e13-bb1d-8987170a5734, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:13.092986 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-27fbc209-4b05-4b4a-8d5c-40d6d87b7e22, external_ids_update=None, columns={'tag': 46342}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:13.093188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.094075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:13.094075 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:13.094075 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:13.094075 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:13.094075 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:13.094075 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:13.094075 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.094075 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:13.095015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 7ff08466-af6d-4e13-bb1d-8987170a5734 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.095015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:13.095015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:16:13.095015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:16:13.095015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:16:13.095015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:16:13.095015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:16:13.095015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:13.095015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.095015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:13.095015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:13.095643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:13.095643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.095643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:13.095643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:13.095643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:13.095643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:13.095643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:13.095643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:13.095643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:13.095643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:13.095643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:13.095643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:13.095643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:13.095643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:13.096346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:13.096346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.096346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:13.096346 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 8a1fb774-5693-4139-8b75-6ab5273228c7 (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:16:13.117466 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, resource={'id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'name': 'public', 'tenant_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'project_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['335d12ce-2d0c-4086-b3f4-2af390c03d70', '5308c10f-28d1-441c-acb2-d1d6dc3d2ece'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-04-28T20:11:05Z', 'updated_at': '2026-04-28T20:11:13Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:13.117957 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:13.118631 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-7dbf713e-d963-4884-ae64-da4ce196e719, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:13.119035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.119860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:13.119860 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:13.119860 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:13.119860 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:13.119860 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:13.119860 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:13.119860 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.119860 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:13.120646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 8a1fb774-5693-4139-8b75-6ab5273228c7 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.120646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:13.120646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:16:13.120646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:16:13.120646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:16:13.120646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:16:13.120646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:16:13.120646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:13.120646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.120646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:13.120646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:13.121239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:13.121239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.121239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:13.121239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:13.121239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:13.121239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:13.121239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:13.121239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:13.121239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:13.121239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:13.121239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:13.121239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:13.121239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:13.121239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:13.121953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:13.121953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.121953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:13.121953 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 51109d7b-e341-4428-8346-5bd0b7c889be (type: security_groups) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:16:13.173781 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.db.ovn_revision_numbers_db [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Successfully bumped revision number for resource 51109d7b-e341-4428-8346-5bd0b7c889be (type: security_groups) to 9
Apr 28 20:16:13.173917 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_for_inconsistencies finished in 0.120 seconds
Apr 28 20:16:13.745821 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:13.746487 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:13.746702 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:13.748320 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:13.748801 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:13.749073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.750196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:13.750196 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:13.750196 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:13.750196 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:13.750196 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:13.750196 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:13.750196 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.750196 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:13.751925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.751925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:13.751925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:13.751925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:13.751925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.751925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:16:13.751925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:13.751925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.751925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:13.751925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:13.751925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:13.751925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.752879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:13.752879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:13.752879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:13.752879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:13.752879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:13.752879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:13.752879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:13.752879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:13.752879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:13.752879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:13.752879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:13.752879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:13.752879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:13.753723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.753723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:13.753723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.753723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:13.753723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:13.753723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:13.753723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:13.753723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:13.753723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.753723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:13.753723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:13.753723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.753723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:13.753723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:13.753723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.754855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:13.754855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:13.754855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.754855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:16:13.754855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:13.754855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.754855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:13.754855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:13.754855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:13.754855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.754855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:13.754855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:13.754855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:13.754855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:13.754855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:13.754855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:13.754855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:13.754855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:13.754855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:13.755855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:13.755855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:13.755855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:13.755855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:13.755855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.755855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:13.794861 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:13.795274 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:13.795440 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:13.795620 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:16:13.796241 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:13.796590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.797432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:13.797432 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:13.797432 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:13.797432 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:13.797432 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:13.797432 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:13.797432 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.797432 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:13.798699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.798699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:16:13.798699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:13.798699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:13.798699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.798699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:13.798699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:13.798699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:13.798699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.798699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:13.798699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:13.798699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:16:13.798699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:13.798699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:16:13.798699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:16:13.799407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:13.799407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:16:13.799407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:13.799407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:16:13.799407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:16:13.799407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:13.799407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:16:13.799407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.799407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:16:13.800034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.800034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:13.800034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:13.800034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:13.800034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.800034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:16:13.800034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:16:13.800034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:16:13.800034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:13.800034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:13.800034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.800034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:13.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:13.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:13.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:13.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:13.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:13.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:13.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:13.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:13.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:13.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:13.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:13.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:13.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:13.801238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:13.801238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:13.801238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.801238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:13.801619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.801619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:13.801619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:13.801619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:13.801619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:13.801619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:13.801619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.801619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:13.801619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:13.801619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.801619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:13.801619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:13.801619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.801619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:13.801619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:13.801619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:16:13.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:16:13.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:16:13.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:13.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:13.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:13.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:13.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:13.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:13.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:13.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:13.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:13.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:13.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:13.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:13.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:13.803093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:13.803093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:13.803093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:13.803093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:13.803093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:13.803093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.803093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:13.805635 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:13.806001 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:13.806173 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:13.806357 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:13.806974 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:13.807330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.808099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:13.808099 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:13.808099 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:13.808099 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:13.808099 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:13.808099 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:13.808099 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.808099 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:13.808921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.808921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:16:13.808921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:13.808921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:13.808921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.808921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:13.808921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:13.808921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:13.808921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.808921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:13.808921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:13.808921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:16:13.808921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:13.808921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:16:13.808921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:16:13.809315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:13.809315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:16:13.809315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:13.809315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:16:13.809315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:16:13.809315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:13.809315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:16:13.809315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.809315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:16:13.809769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.809769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:13.809769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:13.809769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:13.809769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.809769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:16:13.809769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:16:13.809769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:13.809769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:13.809769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.809769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:13.810094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:13.810094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:13.810094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.810094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:13.810094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:13.810094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:13.810094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:13.810094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:13.810094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:13.810094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:13.810094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:13.810094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:13.810094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:13.810094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:13.810468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:13.810468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:13.810468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.810468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:13.810836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.810836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:13.810836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:13.810836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:13.810836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:13.810836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:13.810836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.810836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:13.810836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:13.810836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.810836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:13.810836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:13.810836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.810836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:13.810836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:13.810836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.810836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:16:13.811286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:16:13.811286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:13.811286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:13.811286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.811286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:13.811286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:13.811286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:13.811286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.811286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:13.811286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:13.811286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:13.811286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:13.811286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:13.811286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:13.811286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:13.811286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:13.811286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:13.811286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:13.811286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:13.811775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:13.811775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:13.811775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.811775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:13.813251 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:13.813532 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:13.813648 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:13.813772 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:13.814281 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:13.814728 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:13.815000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.815791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:13.815791 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:13.815791 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:13.815791 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:13.815791 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:13.815791 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:13.815791 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.815791 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:13.816578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.816578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:13.816578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:13.816578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:13.816578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.816578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:16:13.816578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:13.816578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.816578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:13.816578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:13.816578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:13.816578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.817357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:13.817357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:13.817357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:13.817357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:13.817357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:13.817357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:13.817357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:13.817357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:13.817357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:13.817357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:13.817357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:13.817357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:13.817357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:13.817989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.817989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:13.817989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.817989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:13.817989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:13.817989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:13.817989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:13.817989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:13.817989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.817989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:13.817989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:13.817989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.817989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:13.817989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:13.817989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.819062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:13.819062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:13.819062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:13.819062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:16:13.819062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:13.819062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.819062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:13.819062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:13.819062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:13.819062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:13.819062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:13.819062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:13.819062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:13.819062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:13.819062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:13.819062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:13.819062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:13.819062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:13.819062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:13.819943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:13.819943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:13.819943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:13.819943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:13.819943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:13.819943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:17.970620 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:17.971314 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:17.971448 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:17.972689 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:18.001647 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:18.002081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.002914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:18.002914 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:18.002914 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:18.002914 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:18.002914 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:18.002914 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:18.002914 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.002914 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:18.003664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.003664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:18.003664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:18.003664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:18.003664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.003664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:16:18.003664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:18.003664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:18.003664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:18.003664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:18.003664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:18.003664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:18.004371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:18.004371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:18.004371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:18.004371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:18.004371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:18.004371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:18.004371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:18.004371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:18.004371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:18.004371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:18.004371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:18.004371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:18.004371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:18.005104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.005104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:18.005104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.005104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:18.005104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:18.005104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:18.005104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:18.005104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:18.005104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.005104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:18.005104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:18.005104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.005104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:18.005104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:18.005104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.005864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:18.005864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:18.005864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.005864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:16:18.005864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:18.005864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:18.005864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:18.005864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:18.005864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:18.005864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:18.005864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:18.005864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:18.005864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:18.005864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:18.005864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:18.005864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:18.005864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:18.005864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:18.005864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:18.006732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:18.006732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:18.006732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:18.006732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:18.006732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.006732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:18.746465 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:18.747124 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:18.747326 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:18.748223 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:18.748467 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:18.748633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.749377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:18.749377 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:18.749377 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:18.749377 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:18.749377 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:18.749377 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:18.749377 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.749377 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:18.750995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.750995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:18.750995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:18.750995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:18.750995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.750995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:16:18.750995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:18.750995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:18.750995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:18.750995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:18.750995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:18.750995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:18.751833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:18.751833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:18.751833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:18.751833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:18.751833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:18.751833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:18.751833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:18.751833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:18.751833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:18.751833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:18.751833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:18.751833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:18.751833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:18.752630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.752630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:18.752804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.752804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:18.752804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:18.752804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:18.752804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:18.752804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:18.752804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.752804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:18.752804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:18.752804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.752804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:18.752804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:18.752804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.752804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:18.752804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:18.752804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.753773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:16:18.753773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:18.753773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:18.753773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:18.753773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:18.753773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:18.753773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:18.753773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:18.753773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:18.753773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:18.753773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:18.753773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:18.753773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:18.753773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:18.753773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:18.753773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:18.753773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:18.753773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:18.753773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:18.755064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:18.755064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.755064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:18.795466 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:18.795817 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:18.796334 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:18.796334 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:16:18.796733 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:18.797053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.798349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:18.798349 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:18.798349 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:18.798349 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:18.798349 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:18.798349 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:18.798349 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.798349 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:18.799085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.799085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:16:18.799085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:18.799085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:18.799085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:18.799085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:18.799085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:18.799085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:18.799085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:18.799085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:18.799085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:18.799085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:16:18.799085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:18.799085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:16:18.799085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:16:18.799510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:18.799510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:16:18.799510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:18.799510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:16:18.799510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:16:18.799510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:18.799510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:16:18.799510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.799510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:16:18.801694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.801694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:18.801694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:18.801694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:18.801694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.801694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:16:18.801694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:16:18.801694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:16:18.801694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:18.801694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:18.801694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:18.801694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:18.802062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:18.802062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:18.802062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:18.802062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:18.802062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:18.802062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:18.802062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:18.802062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:18.802062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:18.802062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:18.802062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:18.802062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:18.802062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:18.802062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:18.802455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:18.802455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:18.802455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.802455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:18.802455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.802455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:18.802455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:18.802455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:18.802455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:18.802455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:18.802455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.802455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:18.802455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:18.802455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.802958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:18.802958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:18.802958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.802958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:18.802958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:18.802958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.802958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:16:18.802958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:16:18.802958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:16:18.802958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:18.802958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:18.802958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:18.802958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:18.802958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:18.802958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:18.802958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:18.802958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:18.802958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:18.802958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:18.803451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:18.803451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:18.803451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:18.803451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:18.803451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:18.803451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:18.803451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:18.803451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:18.803451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:18.803451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:18.803451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.803451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:18.806080 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:18.806433 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:18.806605 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:18.806797 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:18.807431 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:18.807639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.808342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:18.808342 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:18.808342 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:18.808342 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:18.808342 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:18.808342 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:18.808342 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.808342 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:18.809095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.809095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:16:18.809095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:18.809095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:18.809095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:18.809095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:18.809095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:18.809095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:18.809095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:18.809095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:18.809095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:18.809095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:16:18.809095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:18.809095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:16:18.809095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:16:18.809503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:18.809503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:16:18.809503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:18.809503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:16:18.809503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:16:18.809503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:18.809503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:16:18.809503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.809503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:16:18.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:18.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:18.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:18.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:16:18.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:16:18.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:18.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:18.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:18.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:18.810265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:18.810265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:18.810265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:18.810265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:18.810265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:18.810265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:18.810265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:18.810265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:18.810265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:18.810265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:18.810265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:18.810265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:18.810265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:18.810265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:18.810623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:18.810623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:18.810623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.810623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:18.810974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.810974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:18.810974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:18.810974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:18.810974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:18.810974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:18.810974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.810974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:18.810974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:18.810974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.810974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:18.810974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:18.810974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.810974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:18.810974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:18.810974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.810974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:16:18.811470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:16:18.811470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:18.811470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:18.811470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:18.811470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:18.811470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:18.811470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:18.811470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:18.811470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:18.811470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:18.811470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:18.811470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:18.811470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:18.811470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:18.811470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:18.811470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:18.811470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:18.811470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:18.811470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:18.811955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:18.811955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:18.811955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.811955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:18.813742 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:18.814352 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:18.814352 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:18.814352 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:18.814815 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:18.815212 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:18.815516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.816357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:18.816357 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:18.816357 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:18.816357 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:18.816357 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:18.816357 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:18.816357 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.816357 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:18.817264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.817264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:18.817264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:18.817264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:18.817264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.817264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:16:18.817264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:18.817264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:18.817264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:18.817264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:18.817264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:18.817264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:18.817767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:18.817767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:18.817767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:18.817767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:18.817767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:18.817767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:18.817767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:18.817767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:18.817767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:18.817767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:18.817767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:18.817767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:18.817767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:18.818234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.818234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:18.819018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.819018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:18.819018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:18.819018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:18.819018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:18.819018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:18.819018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.819018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:18.819018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:18.819018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.819018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:18.819018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:18.819018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.819018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:18.819018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:18.819018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:18.819601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:16:18.819601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:18.819601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:18.819601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:18.819601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:18.819601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:18.819601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:18.819601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:18.819601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:18.819601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:18.819601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:18.819601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:18.819601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:18.819601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:18.819601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:18.819601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:18.819601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:18.819601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:18.819601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:18.820359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:18.820359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:18.820359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:22.971253 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:22.971906 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:22.971906 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:22.972463 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:23.008106 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:23.008491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.009316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:23.009316 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:23.009316 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:23.009316 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:23.009316 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:23.009316 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:23.009316 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.009316 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:23.010098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.010098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:23.010098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:23.010098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:23.010098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.010098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:16:23.010098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:23.010098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:23.010098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:23.010098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:23.010098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:23.010098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:23.010655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:23.010655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:23.010655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:23.010655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:23.010655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:23.010655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:23.010655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:23.010655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:23.010655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:23.010655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:23.010655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:23.010655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:23.010655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:23.011221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.011221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:23.011221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.011221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:23.011221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:23.011221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:23.011221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:23.011221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:23.011221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.011221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:23.011221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:23.011221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.011221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:23.011221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:23.011221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.012214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:23.012214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:23.012214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.012214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:16:23.012214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:23.012214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:23.012214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:23.012214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:23.012214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:23.012214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:23.012214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:23.012214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:23.012214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:23.012214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:23.012214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:23.012214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:23.012214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:23.012214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:23.012214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:23.013121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:23.013121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:23.013121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:23.013121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:23.013121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.013121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:23.747320 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:23.747603 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:23.747880 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:23.748841 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:23.749070 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:23.749180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.749772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:23.749772 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:23.749772 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:23.749772 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:23.749772 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:23.749772 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:23.749772 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.749772 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:23.751242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.751242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:23.751242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:23.751242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:23.751242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.751242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:16:23.751242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:23.751242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:23.751242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:23.751242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:23.751242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:23.751242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:23.751597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:23.751597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:23.751597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:23.751597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:23.751597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:23.751597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:23.751597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:23.751597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:23.751597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:23.751597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:23.751597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:23.751597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:23.751597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:23.751975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.751975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:23.753636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.753636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:23.753636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:23.753636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:23.753636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:23.753636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:23.753636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.753636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:23.753636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:23.753636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.753636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:23.753636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:23.753636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.753636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:23.753636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:23.753636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.754104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:16:23.754104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:23.754104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:23.754104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:23.754104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:23.754104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:23.754104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:23.754104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:23.754104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:23.754104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:23.754104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:23.754104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:23.754104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:23.754104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:23.754104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:23.754104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:23.754104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:23.754104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:23.754104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:23.754625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:23.754625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.754625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:23.795786 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:23.796141 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:23.796316 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:23.796491 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:16:23.797107 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:23.797282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.797850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:23.797850 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:23.797850 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:23.797850 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:23.797850 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:23.797850 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:23.797850 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.797850 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:23.799108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.799108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:16:23.799108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:23.799108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:23.799108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:23.799108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:23.799108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:23.799108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:23.799108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:23.799108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:23.799108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:23.799108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:16:23.799108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:23.799108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:16:23.799108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:16:23.800139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:23.800139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:16:23.800139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:23.800139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:16:23.800139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:16:23.800139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:23.800139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:16:23.800139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.800139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:16:23.800535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.800535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:23.800535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:23.800535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:23.800535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.800535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:16:23.800535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:16:23.800535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:16:23.800535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:23.800535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:23.800535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:23.800535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:23.801199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:23.801199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:23.801199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:23.801199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:23.801199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:23.801199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:23.801199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:23.801199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:23.801199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:23.801199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:23.801199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:23.801199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:23.801199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:23.801199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:23.801869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:23.801869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:23.801869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.801869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:23.803336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.803336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:23.803336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:23.803336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:23.803336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:23.803336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:23.803336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.803336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:23.803336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:23.803336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.803336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:23.803336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:23.803336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.803336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:23.803336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:23.803336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.804211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:16:23.804211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:16:23.804211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:16:23.804211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:23.804211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:23.804211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:23.804211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:23.804211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:23.804211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:23.804211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:23.804211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:23.804211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:23.804211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:23.804211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:23.804211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:23.804211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:23.804211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:23.804211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:23.805119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:23.805119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:23.805119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:23.805119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:23.805119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:23.805119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.805119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:23.806770 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:23.806965 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:23.807078 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:23.807206 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:23.809473 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:23.809701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.810229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:23.810229 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:23.810229 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:23.810229 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:23.810229 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:23.810229 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:23.810229 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.810229 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:23.811031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.811031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:16:23.811031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:23.811031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:23.811031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:23.811031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:23.811031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:23.811031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:23.811031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:23.811031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:23.811031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:23.811031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:16:23.811031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:23.811031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:16:23.811031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:16:23.811449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:23.811449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:16:23.811449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:23.811449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:16:23.811449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:16:23.811449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:23.811449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:16:23.811449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.811449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:16:23.811966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.811966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:23.811966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:23.811966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:23.811966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.811966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:16:23.811966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:16:23.811966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:23.811966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:23.811966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:23.811966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:23.812353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:23.812353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:23.812353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:23.812353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:23.812353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:23.812353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:23.812353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:23.812353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:23.812353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:23.812353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:23.812353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:23.812353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:23.812353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:23.812353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:23.812748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:23.812748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:23.812748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.812748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:23.813051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.813051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:23.813051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:23.813051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:23.813051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:23.813051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:23.813051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.813051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:23.813051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:23.813051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.813051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:23.813051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:23.813051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.813051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:23.813051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:23.813051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.813051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:16:23.813587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:16:23.813587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:23.813587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:23.813587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:23.813587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:23.813587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:23.813587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:23.813587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:23.813587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:23.813587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:23.813587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:23.813587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:23.813587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:23.813587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:23.813587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:23.813587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:23.813587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:23.813587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:23.813587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:23.814120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:23.814120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:23.814120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.814120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:23.814120 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:23.814631 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:23.814814 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:23.815034 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:23.815316 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:23.815534 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:23.815694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.816238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:23.816238 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:23.816238 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:23.816238 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:23.816238 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:23.816238 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:23.816238 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.816238 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:23.817091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.817091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:23.817091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:23.817091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:23.817091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.817091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:16:23.817091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:23.817091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:23.817091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:23.817091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:23.817091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:23.817091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:23.817476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:23.817476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:23.817476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:23.817476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:23.817476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:23.817476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:23.817476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:23.817476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:23.817476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:23.817476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:23.817476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:23.817476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:23.817476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:23.817852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.817852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:23.818087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.818087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:23.818087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:23.818087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:23.818087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:23.818087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:23.818087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.818087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:23.818087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:23.818087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.818087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:23.818087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:23.818087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.818087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:23.818087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:23.818087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:23.818547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:16:23.818547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:23.818547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:23.818547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:23.818547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:23.818547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:23.818547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:23.818547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:23.818547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:23.818547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:23.818547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:23.818547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:23.818547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:23.818547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:23.818547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:23.818547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:23.818547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:23.818547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:23.818547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:23.819094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:23.819094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:23.819094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:27.971979 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:27.972658 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:27.972821 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:27.974087 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:28.008946 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:28.009276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.010052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:28.010052 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:28.010052 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:28.010052 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:28.010052 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:28.010052 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:28.010052 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.010052 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:28.011097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.011097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:28.011097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:28.011097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:28.011097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.011097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:16:28.011097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:28.011097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:28.011097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:28.011097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:28.011097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:28.011097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:28.011419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:28.011419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:28.011419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:28.011419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:28.011419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:28.011419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:28.011419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:28.011419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:28.011419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:28.011419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:28.011419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:28.011419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:28.011419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:28.011959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.011959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:28.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:28.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:28.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:28.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:28.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:28.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:28.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:28.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:28.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:28.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:28.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:28.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.013184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:16:28.013184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:28.013184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:28.013184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:28.013184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:28.013184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:28.013184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:28.013184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:28.013184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:28.013184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:28.013184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:28.013184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:28.013184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:28.013184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:28.013184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:28.013184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:28.013184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:28.013184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:28.013184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:28.013666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:28.013666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.013666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:28.747809 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:28.748273 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:28.748410 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:28.749206 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:28.749467 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:28.749616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.750332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:28.750332 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:28.750332 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:28.750332 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:28.750332 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:28.750332 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:28.750332 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.750332 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:28.751525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.751525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:28.751525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:28.751525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:28.751525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.751525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:16:28.751525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:28.751525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:28.751525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:28.751525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:28.751525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:28.751525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:28.752054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:28.752054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:28.752054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:28.752054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:28.752054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:28.752054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:28.752054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:28.752054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:28.752054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:28.752054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:28.752054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:28.752054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:28.752054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:28.752548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.752548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:28.752815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.752815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:28.752815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:28.752815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:28.752815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:28.752815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:28.752815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.752815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:28.752815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:28.752815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.752815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:28.752815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:28.752815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.752815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:28.752815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:28.752815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:16:28.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:28.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:28.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:28.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:28.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:28.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:28.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:28.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:28.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:28.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:28.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:28.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:28.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:28.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:28.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:28.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:28.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:28.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:28.754466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:28.754466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.754466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:28.796134 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:28.796548 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:28.796769 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:28.796971 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:16:28.797498 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:28.797850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.798855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:28.798855 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:28.798855 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:28.798855 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:28.798855 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:28.798855 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:28.798855 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.798855 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:28.800202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.800202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:16:28.800202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:28.800202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:28.800202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:28.800202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:28.800202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:28.800202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:28.800202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:28.800202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:28.800202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:28.800202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:16:28.800202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:28.800202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:16:28.800202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:16:28.800886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:28.800886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:16:28.800886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:28.800886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:16:28.800886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:16:28.800886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:28.800886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:16:28.800886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.800886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:16:28.801861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.801861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:28.801861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:28.801861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:28.801861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.801861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:16:28.801861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:16:28.801861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:16:28.801861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:28.801861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:28.801861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:28.801861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:28.802456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:28.802456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:28.802456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:28.802456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:28.802456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:28.802456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:28.802456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:28.802456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:28.802456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:28.802456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:28.802456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:28.802456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:28.802456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:28.802456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:28.803224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:28.803224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:28.803224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.803224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:28.804016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.804016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:28.804016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:28.804016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:28.804016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:28.804016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:28.804016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.804016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:28.804016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:28.804016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.804016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:28.804016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:28.804016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.804016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:28.804016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:28.804016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.804662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:16:28.804662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:16:28.804662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:16:28.804662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:28.804662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:28.804662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:28.804662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:28.804662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:28.804662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:28.804662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:28.804662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:28.804662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:28.804662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:28.804662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:28.804662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:28.804662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:28.804662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:28.804662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:28.805476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:28.805476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:28.805476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:28.805476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:28.805476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:28.805476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.805476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:28.807042 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:28.807315 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:28.807469 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:28.807622 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:28.808073 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:28.808443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.809186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:28.809186 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:28.809186 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:28.809186 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:28.809186 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:28.809186 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:28.809186 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.809186 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:28.810045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.810045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:16:28.810045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:28.810045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:28.810045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:28.810045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:28.810045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:28.810045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:28.810045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:28.810045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:28.810045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:28.810045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:16:28.810045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:28.810045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:16:28.810045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:16:28.810481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:28.810481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:16:28.810481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:28.810481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:16:28.810481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:16:28.810481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:28.810481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:16:28.810481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.810481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:16:28.811152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.811152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:28.811152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:28.811152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:28.811152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.811152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:16:28.811152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:16:28.811152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:28.811152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:28.811152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:28.811152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:28.811480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:28.811480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:28.811480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:28.811480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:28.811480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:28.811480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:28.811480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:28.811480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:28.811480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:28.811480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:28.811480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:28.811480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:28.811480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:28.811480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:28.811872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:28.811872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:28.811872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.811872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:28.812417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.812417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:28.812417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:28.812417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:28.812417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:28.812417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:28.812417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.812417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:28.812417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:28.812417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.812417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:28.812417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:28.812417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.812417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:28.812417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:28.812417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.812417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:16:28.812904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:16:28.812904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:28.812904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:28.812904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:28.812904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:28.812904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:28.812904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:28.812904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:28.812904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:28.812904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:28.812904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:28.812904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:28.812904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:28.812904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:28.812904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:28.812904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:28.812904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:28.812904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:28.812904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:28.813395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:28.813395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:28.813395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.813395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:28.814588 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:28.814876 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:28.814995 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:28.815416 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:28.815627 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:28.816127 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:28.816371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.817175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:28.817175 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:28.817175 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:28.817175 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:28.817175 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:28.817175 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:28.817175 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.817175 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:28.818142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.818142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:28.818142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:28.818142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:28.818142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.818142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:16:28.818142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:28.818142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:28.818142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:28.818142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:28.818142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:28.818142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:28.818953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:28.818953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:28.818953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:28.818953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:28.818953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:28.818953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:28.818953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:28.818953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:28.818953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:28.818953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:28.818953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:28.818953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:28.818953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:28.819300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.819300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:28.819371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.819371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:28.819371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:28.819371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:28.819371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:28.819371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:28.819371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.819371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:28.819371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:28.819371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.819371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:28.819371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:28.819371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.819371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:28.819371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:28.819371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:28.819814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:16:28.819814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:28.819814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:28.819814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:28.819814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:28.819814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:28.819814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:28.819814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:28.819814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:28.819814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:28.819814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:28.819814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:28.819814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:28.819814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:28.819814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:28.819814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:28.819814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:28.819814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:28.819814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:28.820311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:28.820311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:28.820311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:32.973165 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:32.973827 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:32.973827 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:32.974727 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:33.006085 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:33.006499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.007263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:33.007263 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:33.007263 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:33.007263 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:33.007263 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:33.007263 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:33.007263 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.007263 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:33.008020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.008020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:33.008020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:33.008020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:33.008020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.008020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:16:33.008020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:33.008020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:33.008020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:33.008020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:33.008020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:33.008020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:33.008381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:33.008381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:33.008381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:33.008381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:33.008381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:33.008381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:33.008381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:33.008381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:33.008381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:33.008381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:33.008381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:33.008381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:33.008381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:33.008747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.008747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:33.008871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.008871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:33.008871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:33.008871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:33.008871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:33.008871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:33.008871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.008871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:33.008871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:33.008871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.008871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:33.008871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:33.008871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.008871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:33.008871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:33.008871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.009290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:16:33.009290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:33.009290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:33.009290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:33.009290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:33.009290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:33.009290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:33.009290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:33.009290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:33.009290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:33.009290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:33.009290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:33.009290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:33.009290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:33.009290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:33.009290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:33.009290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:33.009290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:33.009290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:33.009803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:33.009803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.009803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:33.748337 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:33.748809 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:33.748921 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:33.749597 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:33.749914 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:33.750081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.750667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:33.750667 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:33.750667 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:33.750667 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:33.750667 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:33.750667 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:33.750667 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.750667 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:33.751449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.751449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:33.751449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:33.751449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:33.751449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.751449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:16:33.751449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:33.751449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:33.751449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:33.751449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:33.751449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:33.751449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:33.751982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:33.751982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:33.751982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:33.751982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:33.751982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:33.751982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:33.751982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:33.751982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:33.751982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:33.751982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:33.751982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:33.751982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:33.751982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:33.752531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.752531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:33.752531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.752531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:33.752531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:33.752531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:33.752531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:33.752531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:33.752531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.752531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:33.752531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:33.752531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.752531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:33.752531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:33.752531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.753251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:33.753251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:33.753251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.753251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:16:33.753251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:33.753251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:33.753251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:33.753251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:33.753251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:33.753251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:33.753251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:33.753251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:33.753251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:33.753251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:33.753251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:33.753251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:33.753251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:33.753251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:33.753251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:33.754042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:33.754042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:33.754042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:33.754042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:33.754042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.754042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:33.796544 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:33.796846 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:33.796918 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:33.797021 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:16:33.797434 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:33.797761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.798462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:33.798462 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:33.798462 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:33.798462 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:33.798462 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:33.798462 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:33.798462 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.798462 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:33.799024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.799024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:16:33.799024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:33.799024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:33.799024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:33.799024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:33.799024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:33.799024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:33.799024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:33.799024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:33.799024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:33.799024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:16:33.799024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:33.799024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:16:33.799024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:16:33.799824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:33.799824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:16:33.799824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:33.799824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:16:33.799824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:16:33.799824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:33.799824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:16:33.799824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.799824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:16:33.799824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.799824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:33.799824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:33.799824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:33.799824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.800434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:16:33.800434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:16:33.800434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:16:33.800434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:33.800434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:33.800434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:33.800434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:33.800434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:33.800434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:33.800434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:33.800434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:33.800434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:33.800434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:33.801141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:33.801141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:33.801141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:33.801141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:33.801141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:33.801141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:33.801141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:33.801141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:33.801141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:33.801141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:33.801141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.801141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:33.801505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.801505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:33.801505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:33.801505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:33.801505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:33.801505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:33.801505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.801505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:33.801505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:33.801505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.801505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:33.801505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:33.801505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.801505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:33.801505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:33.801505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.802851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:16:33.802851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:16:33.802851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:16:33.802851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:33.802851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:33.802851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:33.802851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:33.802851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:33.802851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:33.802851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:33.802851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:33.802851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:33.802851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:33.802851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:33.802851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:33.802851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:33.802851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:33.802851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:33.803286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:33.803286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:33.803286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:33.803286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:33.803286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:33.803286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.803286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:33.807374 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:33.807594 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:33.807665 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:33.807782 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:33.808083 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:33.808358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.809085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:33.809085 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:33.809085 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:33.809085 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:33.809085 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:33.809085 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:33.809085 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.809085 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:33.810099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.810099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:16:33.810099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:33.810099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:33.810099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:33.810099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:33.810099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:33.810099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:33.810099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:33.810099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:33.810099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:33.810099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:16:33.810099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:33.810099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:16:33.810099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:16:33.810491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:33.810491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:16:33.810491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:33.810491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:16:33.810491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:16:33.810491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:33.810491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:16:33.810491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.810491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:16:33.811441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.811441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:33.811441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:33.811441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:33.811441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.811441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:16:33.811441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:16:33.811441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:33.811441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:33.811441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:33.811441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:33.811764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:33.811764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:33.811764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:33.811764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:33.811764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:33.811764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:33.811764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:33.811764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:33.811764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:33.811764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:33.811764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:33.811764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:33.811764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:33.811764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:33.812130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:33.812130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:33.812130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.812130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:33.813116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.813116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:33.813116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:33.813116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:33.813116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:33.813116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:33.813116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.813116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:33.813116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:33.813116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.813116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:33.813116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:33.813116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.813116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:33.813116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:33.813116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.813116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:16:33.813570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:16:33.813570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:33.813570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:33.813570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:33.813570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:33.813570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:33.813570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:33.813570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:33.813570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:33.813570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:33.813570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:33.813570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:33.813570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:33.813570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:33.813570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:33.813570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:33.813570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:33.813570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:33.813570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:33.814088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:33.814088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:33.814088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.814088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:33.814957 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:33.815361 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:33.815361 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:33.815536 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:33.815963 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:33.816328 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:33.816562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.817302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:33.817302 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:33.817302 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:33.817302 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:33.817302 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:33.817302 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:33.817302 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.817302 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:33.818152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.818152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:33.818152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:33.818152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:33.818152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.818152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:16:33.818152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:33.818152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:33.818152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:33.818152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:33.818152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:33.818152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:33.818734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:33.818734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:33.818734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:33.818734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:33.818734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:33.818734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:33.818734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:33.818734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:33.818734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:33.818734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:33.818734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:33.818734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:33.818734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:33.819351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.819351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:33.819351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.819351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:33.819351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:33.819351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:33.819351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:33.819351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:33.819351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.819351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:33.819351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:33.819351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.819351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:33.819351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:33.819351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.820534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:33.820534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:33.820534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:33.820534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:16:33.820534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:33.820534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:33.820534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:33.820534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:33.820534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:33.820534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:33.820534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:33.820534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:33.820534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:33.820534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:33.820534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:33.820534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:33.820534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:33.820534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:33.820534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:33.821353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:33.821353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:33.821353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:33.821353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:33.821353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:33.821353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:37.974762 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:37.975465 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:37.975572 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:37.976830 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:38.014097 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:38.014273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.014818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:38.014818 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:38.014818 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:38.014818 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:38.014818 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:38.014818 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:38.014818 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.014818 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:38.015552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.015552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:38.015552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:38.015552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:38.015552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.015552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:16:38.015552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:38.015552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:38.015552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:38.015552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:38.015552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:38.015552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:38.016157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:38.016157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:38.016157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:38.016157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:38.016157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:38.016157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:38.016157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:38.016157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:38.016157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:38.016157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:38.016157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:38.016157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:38.016157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:38.016739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.016739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:38.016739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.016739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:38.016739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:38.016739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:38.016739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:38.016739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:38.016739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.016739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:38.016739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:38.016739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.016739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:38.016739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:38.016739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.017604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:38.017604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:38.017604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.017604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:16:38.017604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:38.017604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:38.017604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:38.017604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:38.017604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:38.017604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:38.017604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:38.017604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:38.017604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:38.017604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:38.017604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:38.017604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:38.017604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:38.017604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:38.017604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:38.018472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:38.018472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:38.018472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:38.018472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:38.018472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.018472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:38.749151 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:38.749515 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:38.749668 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:38.750504 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:38.750801 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:38.751018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.751858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:38.751858 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:38.751858 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:38.751858 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:38.751858 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:38.751858 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:38.751858 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.751858 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:38.753221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.753221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:38.753221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:38.753221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:38.753221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.753221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:16:38.753221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:38.753221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:38.753221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:38.753221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:38.753221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:38.753221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:38.753781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:38.753781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:38.753781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:38.753781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:38.753781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:38.753781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:38.753781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:38.753781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:38.753781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:38.753781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:38.753781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:38.753781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:38.753781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:38.754298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.754298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:38.754859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.754859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:38.754859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:38.754859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:38.754859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:38.754859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:38.754859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.754859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:38.754859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:38.754859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.754859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:38.754859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:38.754859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.754859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:38.754859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:38.754859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.756284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:16:38.756284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:38.756284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:38.756284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:38.756284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:38.756284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:38.756284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:38.756284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:38.756284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:38.756284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:38.756284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:38.756284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:38.756284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:38.756284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:38.756284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:38.756284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:38.756284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:38.756284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:38.756284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:38.757022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:38.757022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.757022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:38.797007 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:38.797349 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:38.797516 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:38.797674 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:16:38.798132 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:38.798429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.799513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:38.799513 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:38.799513 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:38.799513 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:38.799513 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:38.799513 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:38.799513 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.799513 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:38.800797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.800797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:16:38.800797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:38.800797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:38.800797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:38.800797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:38.800797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:38.800797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:38.800797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:38.800797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:38.800797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:38.800797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:16:38.800797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:38.800797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:16:38.800797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:16:38.801442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:38.801442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:16:38.801442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:38.801442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:16:38.801442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:16:38.801442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:38.801442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:16:38.801442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.801442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:16:38.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:38.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:38.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:38.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:16:38.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:16:38.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:16:38.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:38.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:38.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:38.802318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:38.803497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:38.803497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:38.803497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:38.803497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:38.803497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:38.803497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:38.803497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:38.803497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:38.803497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:38.803497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:38.803497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:38.803497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:38.803497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:38.803497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:38.804670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:38.804670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:38.804670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.804670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:38.804670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.804670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:38.804670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:38.804670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:38.804670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:38.804670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:38.804670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.804670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:38.804670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:38.804670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.805408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:38.805408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:38.805408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.805408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:38.805408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:38.805408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.805408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:16:38.805408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:16:38.805408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:16:38.805408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:38.805408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:38.805408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:38.805408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:38.805408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:38.805408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:38.805408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:38.805408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:38.805408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:38.805408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:38.806176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:38.806176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:38.806176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:38.806176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:38.806176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:38.806176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:38.806176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:38.806176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:38.806176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:38.806176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:38.806176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.806176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:38.807652 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:38.807808 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:38.807881 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:38.807952 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:38.808191 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:38.808444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.808795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:38.808795 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:38.808795 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:38.808795 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:38.808795 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:38.808795 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:38.808795 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.808795 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:38.809596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.809596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:16:38.809596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:38.809596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:38.809596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:38.809596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:38.809596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:38.809596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:38.809596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:38.809596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:38.809596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:38.809596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:16:38.809596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:38.809596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:16:38.809596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:16:38.810329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:38.810329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:16:38.810329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:38.810329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:16:38.810329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:16:38.810329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:38.810329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:16:38.810329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.810329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:16:38.810960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.810960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:38.810960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:38.810960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:38.810960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.810960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:16:38.810960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:16:38.810960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:38.810960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:38.810960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:38.810960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:38.811283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:38.811283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:38.811283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:38.811283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:38.811283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:38.811283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:38.811283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:38.811283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:38.811283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:38.811283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:38.811283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:38.811283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:38.811283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:38.811283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:38.811665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:38.811665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:38.811665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.811665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:38.812014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.812014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:38.812014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:38.812014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:38.812014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:38.812014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:38.812014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.812014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:38.812014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:38.812014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.812014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:38.812014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:38.812014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.812014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:38.812014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:38.812014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.812014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:16:38.812623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:16:38.812623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:38.812623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:38.812623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:38.812623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:38.812623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:38.812623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:38.812623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:38.812623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:38.812623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:38.812623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:38.812623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:38.812623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:38.812623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:38.812623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:38.812623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:38.812623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:38.812623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:38.812623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:38.813166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:38.813166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:38.813166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.813166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:38.815261 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:38.815449 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:38.815553 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:38.815740 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:38.815943 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:38.816160 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:38.816359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.816805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:38.816805 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:38.816805 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:38.816805 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:38.816805 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:38.816805 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:38.816805 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.816805 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:38.817549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.817549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:38.817549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:38.817549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:38.817549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.817549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:16:38.817549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:38.817549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:38.817549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:38.817549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:38.817549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:38.817549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:38.817920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:38.817920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:38.817920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:38.817920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:38.817920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:38.817920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:38.817920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:38.817920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:38.817920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:38.817920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:38.817920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:38.817920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:38.817920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:38.818258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.818258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:38.818485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.818485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:38.818485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:38.818485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:38.818485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:38.818485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:38.818485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.818485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:38.818485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:38.818485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.818485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:38.818485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:38.818485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.818485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:38.818485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:38.818485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:38.818938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:16:38.818938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:38.818938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:38.818938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:38.818938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:38.818938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:38.818938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:38.818938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:38.818938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:38.818938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:38.818938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:38.818938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:38.818938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:38.818938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:38.818938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:38.818938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:38.818938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:38.818938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:38.818938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:38.819414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:38.819414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:38.819414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:42.975443 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:42.976203 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:42.976385 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:42.977547 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:43.012935 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:43.013364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.014292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:43.014292 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:43.014292 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:43.014292 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:43.014292 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:43.014292 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:43.014292 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.014292 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:43.015072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.015072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:43.015072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:43.015072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:43.015072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.015072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:16:43.015072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:43.015072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:43.015072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:43.015072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:43.015072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:43.015072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:43.015470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:43.015470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:43.015470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:43.015470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:43.015470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:43.015470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:43.015470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:43.015470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:43.015470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:43.015470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:43.015470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:43.015470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:43.015470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:43.015881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.015881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:43.015881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.015881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:43.015881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:43.015881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:43.015881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:43.015881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:43.015881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.015881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:43.015881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:43.015881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.015881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:43.015881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:43.015881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.016307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:43.016307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:43.016307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.016307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:16:43.016307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:43.016307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:43.016307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:43.016307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:43.016307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:43.016307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:43.016307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:43.016307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:43.016307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:43.016307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:43.016307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:43.016307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:43.016307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:43.016307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:43.016307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:43.016860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:43.016860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:43.016860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:43.016860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:43.016860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.016860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:43.750417 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:43.752272 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:43.752476 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:43.753637 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:43.753887 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:43.754555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.754803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:43.754803 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:43.754803 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:43.754803 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:43.754803 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:43.754803 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:43.754803 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.754803 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:43.756454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.756454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:43.756454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:43.756454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:43.756454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.756454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:16:43.756454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:43.756454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:43.756454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:43.756454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:43.756454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:43.756454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:43.757146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:43.757146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:43.757146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:43.757146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:43.757146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:43.757146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:43.757146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:43.757146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:43.757146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:43.757146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:43.757146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:43.757146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:43.757146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:43.757510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.757510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:43.759004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.759004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:43.759004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:43.759004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:43.759004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:43.759004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:43.759004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.759004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:43.759004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:43.759004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.759004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:43.759004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:43.759004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.759004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:43.759004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:43.759004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.759478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:16:43.759478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:43.759478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:43.759478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:43.759478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:43.759478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:43.759478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:43.759478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:43.759478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:43.759478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:43.759478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:43.759478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:43.759478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:43.759478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:43.759478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:43.759478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:43.759478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:43.759478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:43.759478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:43.760013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:43.760013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.760013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:43.798548 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:43.798548 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:43.798548 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:43.798548 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:16:43.798548 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:43.798797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.799355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:43.799355 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:43.799355 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:43.799355 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:43.799355 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:43.799355 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:43.799355 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.799355 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:43.800437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.800437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:16:43.800437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:43.800437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:43.800437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:43.800437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:43.800437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:43.800437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:43.800437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:43.800437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:43.800437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:43.800437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:16:43.800437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:43.800437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:16:43.800437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:16:43.800738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:43.800738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:16:43.800738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:43.800738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:16:43.800738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:16:43.800738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:43.800738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:16:43.800738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.800738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:16:43.801703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.801703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:43.801703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:43.801703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:43.801703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.801703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:16:43.801703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:16:43.801703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:16:43.801703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:43.801703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:43.801703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:43.801703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:43.801966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:43.801966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:43.801966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:43.801966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:43.801966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:43.801966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:43.801966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:43.801966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:43.801966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:43.801966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:43.801966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:43.801966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:43.801966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:43.801966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:43.802307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:43.802307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:43.802307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.802307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:43.802594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.802594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:43.802594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:43.802594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:43.802594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:43.802594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:43.802594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.802594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:43.802594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:43.802594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.802594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:43.802594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:43.802594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.802594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:43.802594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:43.802594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.804905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:16:43.804905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:16:43.804905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:16:43.804905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:43.804905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:43.804905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:43.804905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:43.804905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:43.804905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:43.804905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:43.804905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:43.804905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:43.804905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:43.804905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:43.804905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:43.804905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:43.804905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:43.804905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:43.805237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:43.805237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:43.805237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:43.805237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:43.805237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:43.805237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.805237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:43.807996 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:43.808270 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:43.808392 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:43.808517 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:43.809023 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:43.810849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.810849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:43.810849 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:43.810849 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:43.810849 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:43.810849 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:43.810849 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:43.810849 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.810849 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:43.811894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.811894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:16:43.811894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:43.811894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:43.811894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:43.811894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:43.811894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:43.811894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:43.811894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:43.811894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:43.811894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:43.811894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:16:43.811894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:43.811894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:16:43.811894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:16:43.812212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:43.812212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:16:43.812212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:43.812212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:16:43.812212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:16:43.812212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:43.812212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:16:43.812212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.812212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:16:43.813470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.813470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:43.813470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:43.813470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:43.813470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.813470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:16:43.813470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:16:43.813470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:43.813470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:43.813470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:43.813470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:43.814295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:43.814295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:43.814295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:43.814295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:43.814295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:43.814295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:43.814295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:43.814295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:43.814295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:43.814295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:43.814295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:43.814295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:43.814295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:43.814295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:43.814566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:43.814566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:43.814566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.814566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:43.815285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.815285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:43.815285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:43.815285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:43.815285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:43.815285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:43.815285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.815285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:43.815285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:43.815285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.815285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:43.815285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:43.815285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.815285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:43.815285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:43.815285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.815285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:16:43.815647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:16:43.815647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:43.815647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:43.815647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:43.815647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:43.815647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:43.815647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:43.815647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:43.815647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:43.815647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:43.815647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:43.815647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:43.815647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:43.815647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:43.815647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:43.815647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:43.815647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:43.815647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:43.815647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:43.816265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:43.816265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:43.816265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.816265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:43.816265 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:43.816265 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:43.816265 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:43.817316 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:43.817316 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:43.817316 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:43.817316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.817968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:43.817968 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:43.817968 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:43.817968 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:43.817968 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:43.817968 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:43.817968 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.817968 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:43.819180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.819180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:43.819180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:43.819180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:43.819180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.819180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:16:43.819180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:43.819180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:43.819180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:43.819180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:43.819180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:43.819180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:43.819433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:43.819433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:43.819433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:43.819433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:43.819433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:43.819433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:43.819433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:43.819433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:43.819433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:43.819433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:43.819433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:43.819433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:43.819433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:43.819827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.819827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:43.820696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.820696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:43.820696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:43.820696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:43.820696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:43.820696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:43.820696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.820696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:43.820696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:43.820696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.820696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:43.820696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:43.820696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.820696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:43.820696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:43.820696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:43.821058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:16:43.821058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:43.821058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:43.821058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:43.821058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:43.821058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:43.821058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:43.821058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:43.821058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:43.821058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:43.821058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:43.821058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:43.821058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:43.821058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:43.821058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:43.821058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:43.821058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:43.821058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:43.821058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:43.821703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:43.821703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:43.821703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:47.991387 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:47.993901 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:47.993901 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:47.995181 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:48.031285 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:48.031703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.032602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:48.032602 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:48.032602 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:48.032602 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:48.032602 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:48.032602 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:48.032602 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.032602 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:48.033622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.033622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:48.033622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:48.033622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:48.033622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.033622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:16:48.033622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:48.033622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:48.033622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:48.033622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:48.033622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:48.033622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:48.034219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:48.034219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:48.034219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:48.034219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:48.034219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:48.034219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:48.034219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:48.034219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:48.034219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:48.034219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:48.034219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:48.034219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:48.034219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:48.034867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.034867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:48.034867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.034867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:48.034867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:48.034867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:48.034867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:48.034867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:48.034867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.034867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:48.034867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:48.034867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.034867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:48.034867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:48.034867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:48.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:48.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:16:48.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:48.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:48.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:48.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:48.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:48.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:48.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:48.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:48.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:48.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:48.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:48.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:48.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:48.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:48.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:48.036366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:48.036366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:48.036366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:48.036366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:48.036366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.036366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:48.752715 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:48.753630 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:48.753847 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:48.755134 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:48.755553 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:48.755901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.756917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:48.756917 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:48.756917 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:48.756917 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:48.756917 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:48.756917 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:48.756917 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.756917 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:48.758465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.758465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:48.758465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:48.758465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:48.758465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.758465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:16:48.758465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:48.758465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:48.758465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:48.758465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:48.758465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:48.758465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:48.758982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:48.758982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:48.758982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:48.758982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:48.758982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:48.758982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:48.758982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:48.758982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:48.758982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:48.758982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:48.758982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:48.758982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:48.758982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:48.759405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.759405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:48.760196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.760196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:48.760196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:48.760196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:48.760196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:48.760196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:48.760196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.760196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:48.760196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:48.760196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.760196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:48.760196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:48.760196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.760196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:48.760196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:48.760196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.760764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:16:48.760764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:48.760764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:48.760764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:48.760764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:48.760764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:48.760764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:48.760764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:48.760764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:48.760764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:48.760764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:48.760764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:48.760764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:48.760764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:48.760764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:48.760764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:48.760764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:48.760764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:48.760764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:48.761414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:48.761414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.761414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:48.798500 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:48.799123 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:48.799301 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:48.799482 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:16:48.800271 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:48.800665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.802016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:48.802016 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:48.802016 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:48.802016 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:48.802016 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:48.802016 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:48.802016 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.802016 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:48.802964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.802964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:16:48.802964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:48.802964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:48.802964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:48.802964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:48.802964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:48.802964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:48.802964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:48.802964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:48.802964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:48.802964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:16:48.802964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:48.802964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:16:48.802964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:16:48.803778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:48.803778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:16:48.803778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:48.803778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:16:48.803778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:16:48.803778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:48.803778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:16:48.803778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.803778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:16:48.804215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.804215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:48.804215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:48.804215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:48.804215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.804215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:16:48.804215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:16:48.804215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:16:48.804215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:48.804215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:48.804215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:48.804215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:48.804927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:48.804927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:48.804927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:48.804927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:48.804927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:48.804927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:48.804927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:48.804927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:48.804927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:48.804927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:48.804927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:48.804927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:48.804927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:48.804927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:48.805535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:48.805535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:48.805535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.805535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:48.805879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.805879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:48.805879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:48.805879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:48.805879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:48.805879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:48.805879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.805879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:48.805879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:48.805879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.805879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:48.805879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:48.805879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.805879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:48.805879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:48.805879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.806640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:16:48.806640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:16:48.806640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:16:48.806640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:48.806640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:48.806640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:48.806640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:48.806640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:48.806640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:48.806640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:48.806640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:48.806640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:48.806640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:48.806640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:48.806640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:48.806640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:48.806640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:48.806640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:48.807554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:48.807554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:48.807554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:48.807554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:48.807554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:48.807554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.807554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:48.808536 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:48.809064 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:48.809064 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:48.809213 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:48.809722 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:48.810476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.811215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:48.811215 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:48.811215 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:48.811215 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:48.811215 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:48.811215 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:48.811215 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.811215 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:48.811960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.811960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:16:48.811960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:48.811960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:48.811960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:48.811960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:48.811960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:48.811960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:48.811960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:48.811960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:48.811960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:48.811960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:16:48.811960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:48.811960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:16:48.811960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:16:48.812396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:48.812396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:16:48.812396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:48.812396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:16:48.812396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:16:48.812396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:48.812396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:16:48.812396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.812396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:16:48.812818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.812818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:48.812818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:48.812818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:48.812818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.812818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:16:48.812818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:16:48.812818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:48.812818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:48.812818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:48.812818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:48.813194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:48.813194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:48.813194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:48.813194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:48.813194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:48.813194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:48.813194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:48.813194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:48.813194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:48.813194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:48.813194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:48.813194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:48.813194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:48.813194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:48.813952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:48.813952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:48.813952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.813952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:48.813952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.813952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:48.813952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:48.813952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:48.813952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:48.813952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:48.813952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.813952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:48.813952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:48.813952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.814385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:48.814385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:48.814385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.814385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:48.814385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:48.814385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.814385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:16:48.814385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:16:48.814385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:48.814385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:48.814385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:48.814385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:48.814385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:48.814385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:48.814385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:48.814385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:48.814385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:48.814385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:48.814385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:48.815100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:48.815100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:48.815100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:48.815100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:48.815100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:48.815100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:48.815100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:48.815100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:48.815100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:48.815100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.815100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:48.816286 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:48.816582 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:48.816700 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:48.816857 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:48.817260 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:48.817571 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:48.817803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.818399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:48.818399 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:48.818399 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:48.818399 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:48.818399 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:48.818399 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:48.818399 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.818399 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:48.819218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.819218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:48.819218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:48.819218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:48.819218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.819218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:16:48.819218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:48.819218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:48.819218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:48.819218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:48.819218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:48.819218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:48.819614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:48.819614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:48.819614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:48.819614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:48.819614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:48.819614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:48.819614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:48.819614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:48.819614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:48.819614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:48.819614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:48.819614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:48.819614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:48.820032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.820032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:48.820173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.820173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:48.820173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:48.820173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:48.820173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:48.820173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:48.820173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.820173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:48.820173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:48.820173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.820173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:48.820173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:48.820173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.820173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:48.820173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:48.820173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:48.820655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:16:48.820655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:48.820655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:48.820655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:48.820655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:48.820655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:48.820655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:48.820655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:48.820655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:48.820655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:48.820655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:48.820655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:48.820655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:48.820655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:48.820655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:48.820655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:48.820655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:48.820655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:48.820655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:48.821274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:48.821274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:48.821274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:52.993477 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:52.994233 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:52.994401 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:52.995755 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:53.031795 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:53.032185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.033478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:53.033478 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:53.033478 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:53.033478 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:53.033478 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:53.033478 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:53.033478 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.033478 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:53.033997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.033997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:53.033997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:53.033997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:53.033997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.033997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:16:53.033997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:53.033997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:53.033997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:53.033997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:53.033997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:53.033997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:53.034790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:53.034790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:53.034790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:53.034790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:53.034790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:53.034790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:53.034790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:53.034790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:53.034790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:53.034790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:53.034790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:53.034790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:53.034790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:53.035482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.035482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:53.035482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.035482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:53.035482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:53.035482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:53.035482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:53.035482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:53.035482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.035482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:53.035482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:53.035482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.035482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:53.035482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:53.035482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.036098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:53.036098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:53.036098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.036098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:16:53.036098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:53.036098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:53.036098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:53.036098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:53.036098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:53.036098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:53.036098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:53.036098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:53.036098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:53.036098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:53.036098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:53.036098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:53.036098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:53.036098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:53.036098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:53.036643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:53.036643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:53.036643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:53.036643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:53.036643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.036643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:53.755308 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:53.755308 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:53.755308 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:53.755308 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:53.755759 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:53.756016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.756948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:53.756948 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:53.756948 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:53.756948 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:53.756948 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:53.756948 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:53.756948 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.756948 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:53.758465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.758465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:53.758465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:53.758465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:53.758465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.758465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:16:53.758465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:53.758465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:53.758465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:53.758465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:53.758465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:53.758465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:53.759022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:53.759022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:53.759022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:53.759022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:53.759022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:53.759022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:53.759022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:53.759022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:53.759022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:53.759022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:53.759022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:53.759022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:53.759022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:53.759558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.759558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:53.760552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.760552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:53.760552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:53.760552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:53.760552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:53.760552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:53.760552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.760552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:53.760552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:53.760552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.760552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:53.760552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:53.760552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.760552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:53.760552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:53.760552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.761196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:16:53.761196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:53.761196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:53.761196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:53.761196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:53.761196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:53.761196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:53.761196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:53.761196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:53.761196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:53.761196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:53.761196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:53.761196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:53.761196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:53.761196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:53.761196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:53.761196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:53.761196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:53.761196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:53.761956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:53.761956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.761956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:53.800275 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:53.800552 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:53.800765 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:53.800871 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:16:53.802915 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:53.803060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.803955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:53.803955 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:53.803955 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:53.803955 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:53.803955 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:53.803955 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:53.803955 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.803955 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:53.806271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.806271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:16:53.806271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:53.806271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:53.806271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:53.806271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:53.806271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:53.806271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:53.806271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:53.806271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:53.806271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:53.806271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:16:53.806271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:53.806271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:16:53.806271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:16:53.806815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:53.806815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:16:53.806815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:53.806815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:16:53.806815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:16:53.806815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:53.806815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:16:53.806815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.806815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:16:53.809213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.809213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:53.809213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:53.809213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:53.809213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.809213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:16:53.809213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:16:53.809213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:16:53.809213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:53.809213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:53.809213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:53.809213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:53.809507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:53.809507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:53.809507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:53.809507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:53.809507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:53.809507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:53.809507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:53.809507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:53.809507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:53.809507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:53.809507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:53.809507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:53.809507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:53.809507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:53.809784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:53.809784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:53.809784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.809784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:53.812375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.812375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:53.812375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:53.812375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:53.812375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:53.812375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:53.812375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.812375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:53.812375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:53.812375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.812375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:53.812375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:53.812375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.812375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:53.812375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:53.812375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.812734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:16:53.812734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:16:53.812734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:16:53.812734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:53.812734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:53.812734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:53.812734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:53.812734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:53.812734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:53.812734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:53.812734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:53.812734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:53.812734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:53.812734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:53.812734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:53.812734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:53.812734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:53.812734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:53.813086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:53.813086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:53.813086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:53.813086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:53.813086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:53.813086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.813086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:53.813086 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:53.813546 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:53.814263 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:53.814501 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:53.815153 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:53.815425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.816141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:53.816141 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:53.816141 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:53.816141 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:53.816141 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:53.816141 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:53.816141 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.816141 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:53.817603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.817603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:16:53.817603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:53.817603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:53.817603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:53.817603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:53.817603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:53.817603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:53.817603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:53.817603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:53.817603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:53.817603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:16:53.817603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:53.817603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:16:53.817603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:16:53.818015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:53.818015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:16:53.818015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:53.818015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:16:53.818015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:16:53.818015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:53.818015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:16:53.818015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.818015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:16:53.818015 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:53.819708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.819708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:53.819708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:53.819708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:53.819708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.819708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:16:53.819708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:16:53.819708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:53.819708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:53.819708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:53.819708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:53.820032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:53.820032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:53.820032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:53.820032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:53.820032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:53.820032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:53.820032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:53.820032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:53.820032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:53.820032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:53.820032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:53.820032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:53.820032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:53.820032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:53.820408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:53.820408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:53.820408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.820408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:53.824252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.824252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:53.824252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:53.824252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:53.824252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:53.824252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:53.824252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.824252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:53.824252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:53.824252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.824252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:53.824252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:53.824252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.824252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:53.824252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:53.824252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.824252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:16:53.824728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:16:53.824728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:53.824728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:53.824728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:53.824728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:53.824728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:53.824728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:53.824728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:53.824728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:53.824728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:53.824728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:53.824728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:53.824728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:53.824728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:53.824728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:53.824728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:53.824728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:53.824728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:53.824728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:53.825219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:53.825219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:53.825219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.825219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:53.825219 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:53.825219 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:53.825219 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:53.825693 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:53.826619 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:53.826847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.827814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:53.827814 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:53.827814 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:53.827814 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:53.827814 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:53.827814 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:53.827814 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.827814 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:53.829175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.829175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:53.829175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:53.829175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:53.829175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.829175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:16:53.829175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:53.829175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:53.829175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:53.829175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:53.829175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:53.829175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:53.829442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:53.829442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:53.829442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:53.829442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:53.829442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:53.829442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:53.829442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:53.829442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:53.829442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:53.829442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:53.829442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:53.829442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:53.829442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:53.829756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.829756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:53.830899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.830899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:53.830899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:53.830899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:53.830899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:53.830899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:53.830899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.830899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:53.830899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:53.830899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.830899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:53.830899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:53.830899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.830899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:53.830899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:53.830899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:53.831220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:16:53.831220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:53.831220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:53.831220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:53.831220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:53.831220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:53.831220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:53.831220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:53.831220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:53.831220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:53.831220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:53.831220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:53.831220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:53.831220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:53.831220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:53.831220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:53.831220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:53.831220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:53.831220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:53.831977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:53.831977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:53.831977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:57.994308 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:57.994998 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:57.995070 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:57.996594 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:58.030704 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:58.031204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.032137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:58.032137 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:58.032137 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:58.032137 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:58.032137 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:58.032137 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:58.032137 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.032137 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:58.033222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.033222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:58.033222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:58.033222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:58.033222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.033222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:16:58.033222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:58.033222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:58.033222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:58.033222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:58.033222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:58.033222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:58.033884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:58.033884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:58.033884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:58.033884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:58.033884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:58.033884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:58.033884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:58.033884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:58.033884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:58.033884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:58.033884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:58.033884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:58.033884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:58.034509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.034509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:58.034509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.034509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:58.034509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:58.034509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:58.034509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:58.034509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:58.034509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.034509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:58.034509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:58.034509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.034509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:58.034509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:58.034509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.035214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:58.035214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:58.035214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.035214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:16:58.035214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:58.035214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:58.035214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:58.035214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:58.035214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:58.035214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:58.035214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:58.035214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:58.035214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:58.035214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:58.035214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:58.035214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:58.035214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:58.035214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:58.035214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:58.036057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:58.036057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:58.036057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:58.036057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:58.036057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.036057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:58.754588 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:58.755244 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:58.755441 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:58.756670 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:58.757181 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:58.757436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.758405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:58.758405 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:58.758405 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:58.758405 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:58.758405 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:58.758405 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:58.758405 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.758405 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:58.759971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.759971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:58.759971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:58.759971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:58.759971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.759971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:16:58.759971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:58.759971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:58.759971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:58.759971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:58.759971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:58.759971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:58.760534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:58.760534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:58.760534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:58.760534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:58.760534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:58.760534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:58.760534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:58.760534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:58.760534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:58.760534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:58.760534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:58.760534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:58.760534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:58.761133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.761133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:58.761850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.761850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:58.761850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:58.761850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:58.761850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:58.761850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:58.761850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.761850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:58.761850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:58.761850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.761850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:58.761850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:58.761850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.761850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:58.761850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:58.761850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.762590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:16:58.762590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:58.762590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:58.762590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:58.762590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:58.762590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:58.762590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:58.762590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:58.762590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:58.762590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:58.762590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:58.762590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:58.762590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:58.762590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:58.762590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:58.762590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:58.762590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:58.762590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:58.762590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:58.763417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:58.763417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.763417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:58.800668 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:58.801083 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:58.801269 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:58.801535 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:16:58.802062 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:58.802398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.803185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:58.803185 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:58.803185 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:58.803185 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:58.803185 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:58.803185 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:58.803185 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.803185 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:58.804395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.804395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:16:58.804395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:58.804395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:58.804395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:58.804395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:58.804395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:58.804395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:58.804395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:58.804395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:58.804395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:58.804395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:16:58.804395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:58.804395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:16:58.804395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:16:58.805096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:58.805096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:16:58.805096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:58.805096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:16:58.805096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:16:58.805096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:58.805096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:16:58.805096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.805096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:16:58.805871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.805871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:58.805871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:58.805871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:58.805871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.805871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:16:58.805871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:16:58.805871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:16:58.805871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:58.805871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:58.805871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:58.805871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:58.806458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:58.806458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:58.806458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:58.806458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:58.806458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:58.806458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:58.806458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:58.806458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:58.806458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:58.806458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:58.806458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:58.806458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:58.806458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:58.806458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:58.807068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:58.807068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:58.807068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.807068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:58.807569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.807569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:58.807569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:58.807569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:58.807569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:58.807569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:58.807569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.807569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:58.807569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:58.807569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.807569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:58.807569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:58.807569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.807569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:58.807569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:58.807569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.808291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:16:58.808291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:16:58.808291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:16:58.808291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:58.808291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:58.808291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:58.808291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:58.808291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:58.808291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:58.808291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:58.808291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:58.808291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:58.808291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:58.808291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:58.808291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:58.808291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:58.808291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:58.808291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:58.809060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:58.809060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:58.809060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:58.809060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:58.809060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:58.809060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.809060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:58.813560 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:58.813954 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:58.814424 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:58.814561 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:58.814949 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:58.815233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.815897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:58.815897 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:58.815897 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:58.815897 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:58.815897 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:58.815897 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:58.815897 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.815897 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:58.816735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.816735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:16:58.816735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:58.816735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:58.816735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:58.816735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:58.816735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:58.816735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:58.816735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:58.816735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:16:58.816735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:58.816735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:16:58.816735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:58.816735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:16:58.816735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:16:58.817167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:58.817167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:16:58.817167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:58.817167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:16:58.817167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:16:58.817167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:58.817167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:16:58.817167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.817167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:16:58.817616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.817616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:58.817616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:58.817616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:58.817616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.817616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:16:58.817616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:16:58.817616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:58.817616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:58.817616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:58.817616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:58.817960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:58.817960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:58.817960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:58.817960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:58.817960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:58.817960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:58.817960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:58.817960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:58.817960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:58.817960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:58.817960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:58.817960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:58.817960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:58.817960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:58.818376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:58.818376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:58.818376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.818376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:58.819333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.819333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:58.819333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:58.819333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:58.819333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:58.819333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:58.819333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.819333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:58.819333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:58.819333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.819333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:58.819333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:58.819333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.819333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:58.819333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:58.819333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.819333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:16:58.820097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:16:58.820097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:16:58.820097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:16:58.820097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:58.820097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:58.820097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:58.820097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:58.820097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:58.820097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:58.820097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:58.820097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:58.820097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:58.820097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:58.820097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:58.820097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:58.820097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:58.820097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:58.820097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:58.820097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:58.821050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:58.821050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:58.821050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.821050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:16:58.824692 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:16:58.825132 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:16:58.825315 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:16:58.825390 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:16:58.825893 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:58.826509 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:16:58.826826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.827744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:16:58.827744 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:58.827744 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:16:58.827744 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:16:58.827744 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:58.827744 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:16:58.827744 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.827744 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:16:58.829661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.829661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:16:58.829661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:58.829661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:16:58.829661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.829661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:16:58.829661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:58.829661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:58.829661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:58.829661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:58.829661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:58.829661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:58.830146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:16:58.830146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:58.830146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:16:58.830146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:58.830146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:16:58.830146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:16:58.830146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:58.830146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:16:58.830146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:58.830146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:16:58.830146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:16:58.830146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:58.830146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:16:58.830587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.830587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:16:58.832615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.832615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:16:58.832615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:16:58.832615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:16:58.832615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:16:58.832615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:16:58.832615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.832615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:16:58.832615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:16:58.832615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.832615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:16:58.832615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:16:58.832615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.832615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:16:58.832615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:16:58.832615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:16:58.833355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:16:58.833355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:16:58.833355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:58.833355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:58.833355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:16:58.833355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:16:58.833355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:16:58.833355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:16:58.833355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:16:58.833355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:16:58.833355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:16:58.833355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:16:58.833355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:16:58.833355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:16:58.833355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:16:58.833355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:16:58.833355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:16:58.833355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:16:58.833355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:16:58.833965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:16:58.833965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:16:58.833965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:02.995036 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:02.996602 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:02.996602 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:02.997599 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:02.999716 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=303,neutron:UPDATE=1 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:17:03.038607 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:03.038749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.039379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:03.039379 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:03.039379 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:03.039379 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:03.039379 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:03.039379 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:03.039379 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.039379 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:03.040325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.040325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:03.040325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:03.040325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:03.040325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.040325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:17:03.040325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:03.040325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:03.040325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:03.040325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:03.040325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:03.040325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:03.041035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:03.041035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:03.041035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:03.041035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:03.041035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:03.041035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:03.041035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:03.041035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:03.041035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:03.041035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:03.041035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:03.041035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:03.041035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:03.041571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.041571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:03.041571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.041571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:03.041571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:03.041571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:03.041571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:03.041571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:03.041571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.041571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:03.041571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:03.041571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.041571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:03.041571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:03.041571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.042283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:03.042283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:03.042283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.042283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:17:03.042283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:03.042283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:03.042283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:03.042283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:03.042283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:03.042283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:03.042283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:03.042283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:03.042283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:03.042283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:03.042283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:03.042283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:03.042283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:03.042283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:03.042283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:03.043268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:03.043268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:03.043268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:03.043268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:03.043268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.043268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:03.755303 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:03.756068 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:03.756274 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:03.758369 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:03.758706 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:03.758952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.759830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:03.759830 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:03.759830 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:03.759830 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:03.759830 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:03.759830 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:03.759830 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.759830 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:03.761377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.761377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:03.761377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:03.761377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:03.761377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.761377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:17:03.761377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:03.761377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:03.761377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:03.761377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:03.761377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:03.761377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:03.761944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:03.761944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:03.761944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:03.761944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:03.761944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:03.761944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:03.761944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:03.761944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:03.761944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:03.761944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:03.761944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:03.761944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:03.761944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:03.762480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.762480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:03.762480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.762480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:03.762480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:03.762480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:03.762480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:03.762480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:03.762480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.762480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:03.762480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:03.762480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.762480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:03.762480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:03.762480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.763117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:03.763117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:03.763117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.763117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:17:03.763117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:03.763117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:03.763117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:03.763117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:03.763117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:03.763117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:03.763117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:03.763117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:03.763117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:03.763117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:03.763117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:03.763117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:03.763117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:03.763117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:03.763117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:03.763877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:03.763877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:03.763877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:03.763877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:03.763877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.763877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:03.801185 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:03.801706 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:03.801898 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:03.802094 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:17:03.802651 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:03.803068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.803797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:03.803797 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:03.803797 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:03.803797 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:03.803797 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:03.803797 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:03.803797 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.803797 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:03.805010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.805010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:17:03.805010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:03.805010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:03.805010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:03.805010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:03.805010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:03.805010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:03.805010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:03.805010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:03.805010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:03.805010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:17:03.805010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:03.805010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:17:03.805010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:17:03.805726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:03.805726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:17:03.805726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:03.805726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:17:03.805726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:17:03.805726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:03.805726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:17:03.805726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.805726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:17:03.806550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.806550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:03.806550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:03.806550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:03.806550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.806550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:17:03.806550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:17:03.806550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:17:03.806550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:03.806550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:03.806550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:03.806550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:03.808218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:03.808218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:03.808218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:03.808218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:03.808218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:03.808218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:03.808218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:03.808218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:03.808218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:03.808218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:03.808218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:03.808218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:03.808218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:03.808218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:03.809147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:03.809147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:03.809147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.809147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:03.809147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.809147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:03.809147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:03.809147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:03.809147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:03.809147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:03.809147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.809147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:03.809147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:03.809147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.809790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:03.809790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:03.809790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.809790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:03.809790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:03.809790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.809790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:17:03.809790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:17:03.809790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:17:03.809790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:03.809790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:03.809790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:03.809790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:03.809790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:03.809790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:03.809790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:03.809790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:03.809790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:03.809790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:03.810575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:03.810575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:03.810575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:03.810575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:03.810575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:03.810575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:03.810575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:03.810575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:03.810575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:03.810575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:03.810575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.810575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:03.813968 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:03.814260 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:03.814369 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:03.814494 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:03.814977 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:03.815385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.816106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:03.816106 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:03.816106 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:03.816106 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:03.816106 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:03.816106 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:03.816106 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.816106 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:03.817157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.817157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:17:03.817157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:03.817157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:03.817157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:03.817157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:03.817157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:03.817157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:03.817157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:03.817157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:03.817157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:03.817157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:17:03.817157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:03.817157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:17:03.817157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:17:03.817425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:03.817425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:17:03.817425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:03.817425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:17:03.817425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:17:03.817425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:03.817425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:17:03.817425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.817425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:17:03.817994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.817994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:03.817994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:03.817994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:03.817994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.817994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:17:03.817994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:17:03.817994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:03.817994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:03.817994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:03.817994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:03.818215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:03.818215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:03.818215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:03.818215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:03.818215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:03.818215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:03.818215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:03.818215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:03.818215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:03.818215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:03.818215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:03.818215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:03.818215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:03.818215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:03.818514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:03.818514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:03.818514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.818514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:03.819211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.819211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:03.819211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:03.819211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:03.819211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:03.819211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:03.819211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.819211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:03.819211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:03.819211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.819211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:03.819211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:03.819211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.819211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:03.819211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:03.819211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.819211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:17:03.819525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:17:03.819525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:03.819525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:03.819525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:03.819525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:03.819525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:03.819525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:03.819525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:03.819525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:03.819525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:03.819525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:03.819525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:03.819525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:03.819525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:03.819525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:03.819525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:03.819525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:03.819525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:03.819525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:03.819858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:03.819858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:03.819858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.819858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:03.825252 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:03.825586 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:03.825755 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:03.825889 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:03.826136 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:03.826350 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:03.826487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.826948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:03.826948 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:03.826948 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:03.826948 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:03.826948 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:03.826948 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:03.826948 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.826948 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:03.827812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.827812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:03.827812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:03.827812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:03.827812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.827812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:17:03.827812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:03.827812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:03.827812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:03.827812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:03.827812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:03.827812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:03.828327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:03.828327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:03.828327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:03.828327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:03.828327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:03.828327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:03.828327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:03.828327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:03.828327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:03.828327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:03.828327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:03.828327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:03.828327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:03.828879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.828879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:03.828879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.828879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:03.828879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:03.828879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:03.828879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:03.828879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:03.828879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.828879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:03.828879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:03.828879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.828879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:03.828879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:03.828879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.829464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:03.829464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:03.829464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:03.829464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:17:03.829464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:03.829464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:03.829464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:03.829464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:03.829464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:03.829464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:03.829464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:03.829464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:03.829464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:03.829464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:03.829464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:03.829464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:03.829464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:03.829464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:03.829464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:03.830194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:03.830194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:03.830194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:03.830194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:03.830194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:03.830194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:07.995873 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:07.996465 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:07.996778 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:07.997936 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:08.031405 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:08.031848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.032699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:08.032699 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:08.032699 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:08.032699 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:08.032699 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:08.032699 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:08.032699 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.032699 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:08.033653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.033653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:08.033653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:08.033653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:08.033653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.033653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:17:08.033653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:08.033653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:08.033653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:08.033653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:08.033653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:08.033653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:08.034278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:08.034278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:08.034278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:08.034278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:08.034278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:08.034278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:08.034278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:08.034278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:08.034278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:08.034278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:08.034278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:08.034278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:08.034278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:08.034851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.034851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:08.034952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.034952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:08.034952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:08.034952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:08.034952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:08.034952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:08.034952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.034952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:08.034952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:08.034952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.034952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:08.034952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:08.034952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.034952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:08.034952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:08.034952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.035623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:17:08.035623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:08.035623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:08.035623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:08.035623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:08.035623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:08.035623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:08.035623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:08.035623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:08.035623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:08.035623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:08.035623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:08.035623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:08.035623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:08.035623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:08.035623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:08.035623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:08.035623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:08.035623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:08.036407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:08.036407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.036407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:08.756152 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:08.756818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:08.757021 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:08.758253 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:08.758655 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:08.758948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.759919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:08.759919 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:08.759919 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:08.759919 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:08.759919 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:08.759919 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:08.759919 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.759919 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:08.761631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.761631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:08.761631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:08.761631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:08.761631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.761631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:17:08.761631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:08.761631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:08.761631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:08.761631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:08.761631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:08.761631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:08.762183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:08.762183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:08.762183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:08.762183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:08.762183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:08.762183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:08.762183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:08.762183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:08.762183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:08.762183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:08.762183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:08.762183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:08.762183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:08.762767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.762767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:08.763615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.763615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:08.763615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:08.763615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:08.763615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:08.763615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:08.763615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.763615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:08.763615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:08.763615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.763615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:08.763615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:08.763615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.763615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:08.763615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:08.763615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.764340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:17:08.764340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:08.764340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:08.764340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:08.764340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:08.764340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:08.764340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:08.764340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:08.764340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:08.764340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:08.764340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:08.764340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:08.764340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:08.764340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:08.764340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:08.764340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:08.764340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:08.764340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:08.764340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:08.765150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:08.765150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.765150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:08.802113 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:08.802113 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:08.802345 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:08.802570 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:17:08.803103 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:08.803400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.804330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:08.804330 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:08.804330 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:08.804330 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:08.804330 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:08.804330 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:08.804330 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.804330 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:08.805498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.805498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:17:08.805498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:08.805498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:08.805498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:08.805498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:08.805498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:08.805498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:08.805498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:08.805498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:08.805498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:08.805498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:17:08.805498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:08.805498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:17:08.805498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:17:08.806172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:08.806172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:17:08.806172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:08.806172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:17:08.806172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:17:08.806172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:08.806172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:17:08.806172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.806172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:17:08.806885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.806885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:08.806885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:08.806885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:08.806885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.806885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:17:08.806885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:17:08.806885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:17:08.806885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:08.806885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:08.806885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:08.806885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:08.807461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:08.807461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:08.807461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:08.807461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:08.807461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:08.807461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:08.807461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:08.807461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:08.807461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:08.807461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:08.807461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:08.807461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:08.807461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:08.807461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:08.808091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:08.808091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:08.808091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.808091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:08.809109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.809109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:08.809109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:08.809109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:08.809109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:08.809109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:08.809109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.809109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:08.809109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:08.809109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.809109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:08.809109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:08.809109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.809109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:08.809109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:08.809109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.809725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:17:08.809725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:17:08.809725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:17:08.809725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:08.809725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:08.809725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:08.809725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:08.809725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:08.809725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:08.809725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:08.809725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:08.809725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:08.809725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:08.809725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:08.809725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:08.809725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:08.809725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:08.809725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:08.810219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:08.810219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:08.810219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:08.810219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:08.810219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:08.810219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.810219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:08.814313 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:08.814553 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:08.814628 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:08.814748 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:08.815103 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:08.815370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.816060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:08.816060 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:08.816060 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:08.816060 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:08.816060 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:08.816060 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:08.816060 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.816060 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:08.816752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.816752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:17:08.816752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:08.816752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:08.816752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:08.816752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:08.816752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:08.816752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:08.816752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:08.816752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:08.816752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:08.816752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:17:08.816752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:08.816752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:17:08.816752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:17:08.817157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:08.817157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:17:08.817157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:08.817157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:17:08.817157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:17:08.817157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:08.817157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:17:08.817157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.817157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:17:08.817978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.817978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:08.817978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:08.817978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:08.817978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.817978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:17:08.817978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:17:08.817978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:08.817978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:08.817978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:08.817978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:08.818748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:08.818748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:08.818748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:08.818748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:08.818748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:08.818748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:08.818748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:08.818748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:08.818748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:08.818748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:08.818748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:08.818748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:08.818748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:08.818748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:08.819346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:08.819346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:08.819346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.819346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:08.819346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.819346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:08.819346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:08.819346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:08.819346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:08.819346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:08.819346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.819346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:08.819346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:08.819346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.820022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:08.820022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:08.820022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.820022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:08.820022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:08.820022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.820022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:17:08.820022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:17:08.820022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:08.820022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:08.820022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:08.820022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:08.820022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:08.820022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:08.820022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:08.820022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:08.820022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:08.820022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:08.820022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:08.820868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:08.820868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:08.820868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:08.820868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:08.820868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:08.820868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:08.820868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:08.820868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:08.820868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:08.820868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.820868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:08.825832 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:08.826149 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:08.826257 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:08.826426 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:08.826908 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:08.827241 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:08.827487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.828234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:08.828234 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:08.828234 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:08.828234 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:08.828234 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:08.828234 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:08.828234 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.828234 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:08.829118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.829118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:08.829118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:08.829118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:08.829118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.829118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:17:08.829118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:08.829118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:08.829118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:08.829118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:08.829118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:08.829118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:08.829487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:08.829487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:08.829487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:08.829487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:08.829487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:08.829487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:08.829487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:08.829487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:08.829487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:08.829487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:08.829487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:08.829487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:08.829487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:08.829871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.829871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:08.830151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.830151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:08.830151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:08.830151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:08.830151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:08.830151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:08.830151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.830151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:08.830151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:08.830151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.830151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:08.830151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:08.830151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.830151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:08.830151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:08.830151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:08.830582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:17:08.830582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:08.830582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:08.830582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:08.830582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:08.830582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:08.830582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:08.830582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:08.830582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:08.830582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:08.830582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:08.830582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:08.830582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:08.830582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:08.830582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:08.830582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:08.830582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:08.830582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:08.830582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:08.831117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:08.831117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:08.831117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:12.996550 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:12.997396 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:12.997396 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:12.998646 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:13.032488 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:13.032942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.033823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:13.033823 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:13.033823 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:13.033823 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:13.033823 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:13.033823 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:13.033823 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.033823 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:13.034587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.034587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:13.034587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:13.034587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:13.034587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.034587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:17:13.034587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:13.034587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:13.034587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:13.034587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:13.034587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:13.034587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:13.035276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:13.035276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:13.035276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:13.035276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:13.035276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:13.035276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:13.035276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:13.035276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:13.035276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:13.035276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:13.035276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:13.035276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:13.035276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:13.035953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.035953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:13.035953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.035953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:13.035953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:13.035953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:13.035953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:13.035953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:13.035953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.035953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:13.035953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:13.035953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.035953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:13.035953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:13.035953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.036725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:13.036725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:13.036725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.036725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:17:13.036725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:13.036725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:13.036725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:13.036725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:13.036725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:13.036725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:13.036725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:13.036725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:13.036725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:13.036725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:13.036725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:13.036725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:13.036725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:13.036725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:13.036725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:13.037842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:13.037842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:13.037842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:13.037842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:13.037842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.037842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:13.757191 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:13.758002 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:13.758246 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:13.759786 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:13.760147 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:13.760405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.761346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:13.761346 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:13.761346 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:13.761346 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:13.761346 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:13.761346 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:13.761346 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.761346 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:13.762952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.762952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:13.762952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:13.762952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:13.762952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.762952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:17:13.762952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:13.762952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:13.762952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:13.762952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:13.762952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:13.762952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:13.763586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:13.763586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:13.763586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:13.763586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:13.763586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:13.763586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:13.763586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:13.763586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:13.763586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:13.763586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:13.763586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:13.763586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:13.763586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:13.764228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.764228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:13.764658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.764658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:13.764658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:13.764658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:13.764658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:13.764658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:13.764658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.764658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:13.764658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:13.764658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.764658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:13.764658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:13.764658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.764658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:13.764658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:13.764658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.765745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:17:13.765745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:13.765745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:13.765745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:13.765745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:13.765745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:13.765745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:13.765745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:13.765745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:13.765745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:13.765745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:13.765745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:13.765745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:13.765745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:13.765745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:13.765745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:13.765745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:13.765745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:13.765745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:13.766269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:13.766269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.766269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:13.802573 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:13.803242 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:13.803420 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:13.803606 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:17:13.804085 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:13.804300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.804874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:13.804874 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:13.804874 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:13.804874 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:13.804874 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:13.804874 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:13.804874 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.804874 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:13.805706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.805706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:17:13.805706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:13.805706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:13.805706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:13.805706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:13.805706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:13.805706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:13.805706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:13.805706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:13.805706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:13.805706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:17:13.805706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:13.805706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:17:13.805706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:17:13.806814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:13.806814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:17:13.806814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:13.806814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:17:13.806814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:17:13.806814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:13.806814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:17:13.806814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.806814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:17:13.806814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.806814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:13.806814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:13.806814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:13.806814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.807237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:17:13.807237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:17:13.807237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:17:13.807237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:13.807237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:13.807237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:13.807237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:13.807237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:13.807237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:13.807237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:13.807237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:13.807237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:13.807237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:13.807649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:13.807649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:13.807649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:13.807649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:13.807649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:13.807649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:13.807649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:13.807649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:13.807649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:13.807649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:13.807649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.807649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:13.808176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.808176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:13.808176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:13.808176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:13.808176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:13.808176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:13.808176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.808176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:13.808176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:13.808176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.808176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:13.808176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:13.808176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.808176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:13.808176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:13.808176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.808976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:17:13.808976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:17:13.808976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:17:13.808976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:13.808976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:13.808976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:13.808976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:13.808976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:13.808976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:13.808976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:13.808976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:13.808976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:13.808976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:13.808976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:13.808976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:13.808976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:13.808976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:13.808976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:13.809753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:13.809753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:13.809753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:13.809753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:13.809753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:13.809753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.809753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:13.814979 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:13.815318 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:13.815431 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:13.815558 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:13.815938 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:13.816238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.816790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:13.816790 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:13.816790 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:13.816790 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:13.816790 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:13.816790 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:13.816790 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.816790 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:13.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:17:13.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:13.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:13.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:13.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:13.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:13.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:13.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:13.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:13.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:13.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:17:13.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:13.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:17:13.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:17:13.818023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:13.818023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:17:13.818023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:13.818023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:17:13.818023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:17:13.818023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:13.818023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:17:13.818023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.818023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:17:13.818403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.818403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:13.818403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:13.818403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:13.818403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.818403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:17:13.818403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:17:13.818403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:13.818403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:13.818403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:13.818403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:13.818804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:13.818804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:13.818804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:13.818804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:13.818804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:13.818804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:13.818804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:13.818804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:13.818804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:13.818804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:13.818804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:13.818804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:13.818804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:13.818804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:13.819236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:13.819236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:13.819236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.819236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:13.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:13.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:13.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:13.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:13.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:13.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:13.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:13.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:13.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:13.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:13.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:13.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:17:13.820050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:17:13.820050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:13.820050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:13.820050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:13.820050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:13.820050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:13.820050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:13.820050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:13.820050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:13.820050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:13.820050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:13.820050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:13.820050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:13.820050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:13.820050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:13.820050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:13.820050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:13.820050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:13.820050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:13.820561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:13.820561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:13.820561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.820561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:13.826232 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:13.826564 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:13.826696 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:13.826852 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:13.827369 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:13.827817 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:13.828101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.828931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:13.828931 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:13.828931 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:13.828931 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:13.828931 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:13.828931 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:13.828931 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.828931 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:13.829821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.829821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:13.829821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:13.829821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:13.829821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.829821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:17:13.829821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:13.829821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:13.829821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:13.829821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:13.829821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:13.829821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:13.830306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:13.830306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:13.830306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:13.830306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:13.830306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:13.830306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:13.830306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:13.830306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:13.830306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:13.830306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:13.830306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:13.830306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:13.830306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:13.830774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.830774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:13.830865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.830865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:13.830865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:13.830865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:13.830865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:13.830865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:13.830865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.830865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:13.830865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:13.830865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.830865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:13.830865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:13.830865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.830865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:13.830865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:13.830865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:13.831404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:17:13.831404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:13.831404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:13.831404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:13.831404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:13.831404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:13.831404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:13.831404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:13.831404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:13.831404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:13.831404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:13.831404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:13.831404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:13.831404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:13.831404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:13.831404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:13.831404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:13.831404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:13.831404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:13.831971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:13.831971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:13.831971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:17.997213 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:17.998121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:17.998121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:17.999208 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:18.031672 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:18.032132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.033181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:18.033181 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:18.033181 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:18.033181 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:18.033181 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:18.033181 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:18.033181 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.033181 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:18.034079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.034079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:18.034079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:18.034079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:18.034079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.034079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:17:18.034079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:18.034079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:18.034079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:18.034079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:18.034079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:18.034079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:18.034696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:18.034696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:18.034696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:18.034696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:18.034696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:18.034696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:18.034696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:18.034696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:18.034696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:18.034696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:18.034696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:18.034696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:18.034696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:18.035236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.035236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:18.035236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.035236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:18.035236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:18.035236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:18.035236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:18.035236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:18.035236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.035236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:18.035236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:18.035236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.035236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:18.035236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:18.035236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.036030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:18.036030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:18.036030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.036030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:17:18.036030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:18.036030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:18.036030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:18.036030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:18.036030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:18.036030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:18.036030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:18.036030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:18.036030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:18.036030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:18.036030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:18.036030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:18.036030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:18.036030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:18.036030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:18.036939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:18.036939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:18.036939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:18.036939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:18.036939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.036939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:18.758159 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:18.758689 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:18.758911 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:18.760131 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:18.760569 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:18.760962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.761966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:18.761966 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:18.761966 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:18.761966 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:18.761966 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:18.761966 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:18.761966 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.761966 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:18.763465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.763465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:18.763465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:18.763465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:18.763465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.763465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:17:18.763465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:18.763465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:18.763465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:18.763465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:18.763465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:18.763465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:18.764075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:18.764075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:18.764075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:18.764075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:18.764075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:18.764075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:18.764075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:18.764075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:18.764075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:18.764075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:18.764075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:18.764075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:18.764075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:18.764598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.764598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:18.765118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.765118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:18.765118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:18.765118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:18.765118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:18.765118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:18.765118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.765118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:18.765118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:18.765118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.765118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:18.765118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:18.765118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.765118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:18.765118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:18.765118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.765835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:17:18.765835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:18.765835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:18.765835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:18.765835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:18.765835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:18.765835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:18.765835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:18.765835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:18.765835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:18.765835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:18.765835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:18.765835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:18.765835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:18.765835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:18.765835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:18.765835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:18.765835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:18.765835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:18.766640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:18.766640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.766640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:18.803290 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:18.803800 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:18.804036 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:18.804294 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:17:18.804917 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:18.805222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.806003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:18.806003 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:18.806003 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:18.806003 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:18.806003 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:18.806003 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:18.806003 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.806003 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:18.807457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.807457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:17:18.807457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:18.807457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:18.807457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:18.807457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:18.807457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:18.807457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:18.807457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:18.807457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:18.807457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:18.807457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:17:18.807457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:18.807457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:17:18.807457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:17:18.808181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:18.808181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:17:18.808181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:18.808181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:17:18.808181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:17:18.808181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:18.808181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:17:18.808181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.808181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:17:18.809198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.809198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:18.809198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:18.809198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:18.809198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.809198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:17:18.809198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:17:18.809198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:17:18.809198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:18.809198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:18.809198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:18.809198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:18.809705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:18.809705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:18.809705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:18.809705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:18.809705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:18.809705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:18.809705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:18.809705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:18.809705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:18.809705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:18.809705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:18.809705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:18.809705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:18.809705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:18.810322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:18.810322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:18.810322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.810322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:18.811434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.811434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:18.811434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:18.811434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:18.811434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:18.811434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:18.811434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.811434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:18.811434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:18.811434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.811434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:18.811434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:18.811434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.811434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:18.811434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:18.811434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.812151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:17:18.812151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:17:18.812151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:17:18.812151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:18.812151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:18.812151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:18.812151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:18.812151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:18.812151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:18.812151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:18.812151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:18.812151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:18.812151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:18.812151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:18.812151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:18.812151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:18.812151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:18.812151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:18.812946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:18.812946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:18.812946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:18.812946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:18.812946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:18.812946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.812946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:18.815458 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:18.815710 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:18.815846 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:18.816043 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:18.816449 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:18.816911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.817555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:18.817555 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:18.817555 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:18.817555 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:18.817555 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:18.817555 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:18.817555 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.817555 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:18.818356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.818356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:17:18.818356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:18.818356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:18.818356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:18.818356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:18.818356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:18.818356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:18.818356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:18.818356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:18.818356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:18.818356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:17:18.818356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:18.818356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:17:18.818356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:17:18.818778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:18.818778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:17:18.818778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:18.818778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:17:18.818778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:17:18.818778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:18.818778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:17:18.818778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.818778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:17:18.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:18.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:18.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:18.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:17:18.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:17:18.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:18.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:18.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:18.819442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:18.819773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:18.819773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:18.819773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:18.819773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:18.819773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:18.819773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:18.819773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:18.819773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:18.819773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:18.819773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:18.819773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:18.819773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:18.819773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:18.819773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:18.820139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:18.820139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:18.820139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.820139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:18.820800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.820800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:18.820800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:18.820800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:18.820800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:18.820800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:18.820800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.820800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:18.820800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:18.820800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.820800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:18.820800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:18.820800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.820800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:18.820800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:18.820800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.820800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:17:18.821274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:17:18.821274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:18.821274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:18.821274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:18.821274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:18.821274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:18.821274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:18.821274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:18.821274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:18.821274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:18.821274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:18.821274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:18.821274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:18.821274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:18.821274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:18.821274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:18.821274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:18.821274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:18.821274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:18.821755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:18.821755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:18.821755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.821755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:18.826587 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:18.826849 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:18.827008 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:18.827192 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:18.827607 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:18.828003 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:18.828235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.828943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:18.828943 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:18.828943 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:18.828943 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:18.828943 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:18.828943 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:18.828943 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.828943 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:18.829812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.829812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:18.829812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:18.829812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:18.829812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.829812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:17:18.829812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:18.829812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:18.829812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:18.829812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:18.829812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:18.829812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:18.830331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:18.830331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:18.830331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:18.830331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:18.830331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:18.830331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:18.830331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:18.830331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:18.830331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:18.830331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:18.830331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:18.830331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:18.830331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:18.830913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.830913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:18.830913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.830913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:18.830913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:18.830913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:18.830913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:18.830913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:18.830913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.830913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:18.830913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:18.830913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.830913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:18.830913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:18.830913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.831543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:18.831543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:18.831543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:18.831543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:17:18.831543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:18.831543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:18.831543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:18.831543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:18.831543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:18.831543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:18.831543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:18.831543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:18.831543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:18.831543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:18.831543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:18.831543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:18.831543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:18.831543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:18.831543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:18.832336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:18.832336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:18.832336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:18.832336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:18.832336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:18.832336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:22.997857 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:22.998332 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:22.998472 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:22.999779 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:23.032379 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:23.032848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.033728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:23.033728 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:23.033728 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:23.033728 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:23.033728 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:23.033728 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:23.033728 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.033728 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:23.034502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.034502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:23.034502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:23.034502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:23.034502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.034502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:17:23.034502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:23.034502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:23.034502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:23.034502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:23.034502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:23.034502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:23.035708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:23.035708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:23.035708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:23.035708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:23.035708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:23.035708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:23.035708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:23.035708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:23.035708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:23.035708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:23.035708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:23.035708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:23.035708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:23.036203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.036203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:23.036203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.036203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:23.036203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:23.036203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:23.036203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:23.036203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:23.036203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.036203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:23.036203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:23.036203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.036203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:23.036203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:23.036203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.036848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:23.036848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:23.036848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.036848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:17:23.036848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:23.036848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:23.036848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:23.036848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:23.036848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:23.036848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:23.036848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:23.036848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:23.036848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:23.036848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:23.036848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:23.036848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:23.036848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:23.036848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:23.036848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:23.037603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:23.037603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:23.037603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:23.037603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:23.037603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.037603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:23.758721 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:23.759341 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:23.759595 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:23.761093 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:23.761513 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:23.761770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.763002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:23.763002 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:23.763002 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:23.763002 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:23.763002 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:23.763002 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:23.763002 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.763002 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:23.764470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.764470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:23.764470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:23.764470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:23.764470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.764470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:17:23.764470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:23.764470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:23.764470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:23.764470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:23.764470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:23.764470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:23.765565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:23.765565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:23.765565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:23.765565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:23.765565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:23.765565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:23.765565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:23.765565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:23.765565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:23.765565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:23.765565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:23.765565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:23.765565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:23.766669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.766669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:23.766669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.766669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:23.766669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:23.766669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:23.766669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:23.766669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:23.766669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.766669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:23.766669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:23.766669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.766669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:23.766669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:23.766669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.767614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:23.767614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:23.767614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.767614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:17:23.767614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:23.767614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:23.767614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:23.767614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:23.767614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:23.767614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:23.767614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:23.767614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:23.767614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:23.767614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:23.767614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:23.767614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:23.767614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:23.767614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:23.767614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:23.768294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:23.768294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:23.768294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:23.768294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:23.768294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.768294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:23.803755 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:23.804059 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:23.804231 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:23.804396 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:17:23.804968 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:23.805321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.806186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:23.806186 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:23.806186 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:23.806186 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:23.806186 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:23.806186 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:23.806186 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.806186 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:23.807458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.807458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:17:23.807458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:23.807458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:23.807458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:23.807458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:23.807458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:23.807458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:23.807458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:23.807458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:23.807458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:23.807458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:17:23.807458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:23.807458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:17:23.807458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:17:23.808956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:23.808956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:17:23.808956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:23.808956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:17:23.808956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:17:23.808956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:23.808956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:17:23.808956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.808956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:17:23.808956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.808956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:23.808956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:23.808956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:23.808956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.810467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:17:23.810467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:17:23.810467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:17:23.810467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:23.810467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:23.810467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:23.810467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:23.810467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:23.810467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:23.810467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:23.810467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:23.810467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:23.810467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:23.811186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:23.811186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:23.811186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:23.811186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:23.811186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:23.811186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:23.811186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:23.811186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:23.811186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:23.811186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:23.811186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.811186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:23.812602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.812602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:23.812602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:23.812602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:23.812602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:23.812602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:23.812602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.812602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:23.812602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:23.812602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.812602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:23.812602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:23.812602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.812602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:23.812602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:23.812602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.813660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:17:23.813660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:17:23.813660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:17:23.813660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:23.813660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:23.813660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:23.813660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:23.813660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:23.813660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:23.813660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:23.813660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:23.813660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:23.813660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:23.813660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:23.813660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:23.813660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:23.813660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:23.813660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:23.814887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:23.814887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:23.814887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:23.814887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:23.814887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:23.814887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.814887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:23.815901 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:23.816171 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:23.816247 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:23.816382 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:23.816913 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:23.817304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.818063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:23.818063 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:23.818063 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:23.818063 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:23.818063 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:23.818063 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:23.818063 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.818063 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:23.818992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.818992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:17:23.818992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:23.818992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:23.818992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:23.818992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:23.818992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:23.818992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:23.818992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:23.818992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:23.818992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:23.818992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:17:23.818992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:23.818992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:17:23.818992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:17:23.819481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:23.819481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:17:23.819481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:23.819481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:17:23.819481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:17:23.819481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:23.819481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:17:23.819481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.819481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:17:23.820090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.820090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:23.820090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:23.820090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:23.820090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.820090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:17:23.820090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:17:23.820090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:23.820090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:23.820090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:23.820090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:23.820899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:23.820899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:23.820899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:23.820899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:23.820899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:23.820899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:23.820899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:23.820899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:23.820899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:23.820899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:23.820899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:23.820899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:23.820899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:23.820899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:23.821639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:23.821639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:23.821639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.821639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:23.821639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.821639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:23.821639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:23.821639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:23.821639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:23.821639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:23.821639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.821639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:23.821639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:23.821639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.822074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:23.822074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:23.822074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.822074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:23.822074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:23.822074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.822074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:17:23.822074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:17:23.822074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:23.822074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:23.822074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:23.822074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:23.822074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:23.822074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:23.822074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:23.822074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:23.822074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:23.822074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:23.822074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:23.822575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:23.822575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:23.822575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:23.822575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:23.822575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:23.822575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:23.822575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:23.822575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:23.822575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:23.822575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.822575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:23.826849 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:23.827181 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:23.827298 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:23.827525 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:23.828119 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:23.828524 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:23.828821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.829577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:23.829577 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:23.829577 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:23.829577 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:23.829577 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:23.829577 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:23.829577 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.829577 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:23.830545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.830545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:23.830545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:23.830545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:23.830545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.830545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:17:23.830545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:23.830545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:23.830545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:23.830545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:23.830545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:23.830545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:23.830911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:23.830911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:23.830911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:23.830911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:23.830911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:23.830911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:23.830911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:23.830911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:23.830911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:23.830911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:23.830911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:23.830911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:23.830911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:23.831260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.831260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:23.831742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.831742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:23.831742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:23.831742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:23.831742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:23.831742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:23.831742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.831742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:23.831742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:23.831742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.831742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:23.831742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:23.831742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.831742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:23.831742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:23.831742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:23.832224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:17:23.832224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:23.832224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:23.832224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:23.832224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:23.832224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:23.832224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:23.832224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:23.832224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:23.832224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:23.832224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:23.832224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:23.832224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:23.832224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:23.832224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:23.832224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:23.832224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:23.832224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:23.832224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:23.832850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:23.832850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:23.832850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:27.998512 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:27.999271 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:27.999271 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:28.000523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:28.033634 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:28.033796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.034248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:28.034248 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:28.034248 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:28.034248 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:28.034248 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:28.034248 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:28.034248 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.034248 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:28.035022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.035022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:28.035022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:28.035022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:28.035022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.035022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:17:28.035022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:28.035022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:28.035022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:28.035022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:28.035022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:28.035022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:28.035629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:28.035629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:28.035629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:28.035629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:28.035629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:28.035629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:28.035629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:28.035629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:28.035629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:28.035629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:28.035629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:28.035629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:28.035629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:28.036361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.036361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:28.036361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.036361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:28.036361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:28.036361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:28.036361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:28.036361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:28.036361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.036361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:28.036361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:28.036361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.036361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:28.036361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:28.036361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.037080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:28.037080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:28.037080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.037080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:17:28.037080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:28.037080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:28.037080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:28.037080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:28.037080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:28.037080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:28.037080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:28.037080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:28.037080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:28.037080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:28.037080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:28.037080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:28.037080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:28.037080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:28.037080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:28.037960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:28.037960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:28.037960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:28.037960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:28.037960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.037960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:28.759494 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:28.759972 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:28.760150 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:28.762012 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:28.762012 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:28.762312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.763913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:28.763913 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:28.763913 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:28.763913 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:28.763913 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:28.763913 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:28.763913 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.763913 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:28.765618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.765618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:28.765618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:28.765618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:28.765618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.765618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:17:28.765618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:28.765618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:28.765618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:28.765618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:28.765618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:28.765618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:28.766120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:28.766120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:28.766120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:28.766120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:28.766120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:28.766120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:28.766120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:28.766120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:28.766120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:28.766120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:28.766120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:28.766120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:28.766120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:28.766521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.766521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:28.767899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.767899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:28.767899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:28.767899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:28.767899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:28.767899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:28.767899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.767899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:28.767899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:28.767899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.767899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:28.767899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:28.767899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.767899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:28.767899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:28.767899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.768423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:17:28.768423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:28.768423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:28.768423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:28.768423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:28.768423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:28.768423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:28.768423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:28.768423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:28.768423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:28.768423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:28.768423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:28.768423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:28.768423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:28.768423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:28.768423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:28.768423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:28.768423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:28.768423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:28.769022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:28.769022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.769022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:28.804150 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:28.804445 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:28.804613 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:28.804834 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:17:28.805193 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:28.805352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.805862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:28.805862 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:28.805862 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:28.805862 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:28.805862 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:28.805862 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:28.805862 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.805862 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:28.807057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.807057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:17:28.807057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:28.807057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:28.807057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:28.807057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:28.807057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:28.807057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:28.807057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:28.807057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:28.807057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:28.807057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:17:28.807057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:28.807057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:17:28.807057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:17:28.807370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:28.807370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:17:28.807370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:28.807370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:17:28.807370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:17:28.807370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:28.807370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:17:28.807370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.807370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:17:28.808349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.808349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:28.808349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:28.808349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:28.808349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.808349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:17:28.808349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:17:28.808349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:17:28.808349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:28.808349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:28.808349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:28.808349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:28.808613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:28.808613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:28.808613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:28.808613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:28.808613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:28.808613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:28.808613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:28.808613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:28.808613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:28.808613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:28.808613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:28.808613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:28.808613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:28.808613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:28.808892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:28.808892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:28.808892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.808892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:28.810186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.810186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:28.810186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:28.810186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:28.810186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:28.810186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:28.810186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.810186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:28.810186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:28.810186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.810186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:28.810186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:28.810186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.810186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:28.810186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:28.810186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.810514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:17:28.810514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:17:28.810514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:17:28.810514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:28.810514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:28.810514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:28.810514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:28.810514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:28.810514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:28.810514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:28.810514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:28.810514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:28.810514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:28.810514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:28.810514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:28.810514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:28.810514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:28.810514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:28.810885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:28.810885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:28.810885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:28.810885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:28.810885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:28.810885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.810885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:28.816043 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:28.816229 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:28.816336 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:28.816466 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:28.816915 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:28.817244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.817663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:28.817663 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:28.817663 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:28.817663 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:28.817663 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:28.817663 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:28.817663 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.817663 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:28.818448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.818448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:17:28.818448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:28.818448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:28.818448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:28.818448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:28.818448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:28.818448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:28.818448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:28.818448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:28.818448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:28.818448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:17:28.818448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:28.818448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:17:28.818448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:17:28.818750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:28.818750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:17:28.818750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:28.818750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:17:28.818750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:17:28.818750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:28.818750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:17:28.818750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.818750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:17:28.819341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.819341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:28.819341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:28.819341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:28.819341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.819341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:17:28.819341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:17:28.819341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:28.819341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:28.819341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:28.819341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:28.819573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:28.819573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:28.819573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:28.819573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:28.819573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:28.819573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:28.819573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:28.819573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:28.819573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:28.819573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:28.819573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:28.819573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:28.819573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:28.819573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:28.819855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:28.819855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:28.819855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.819855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:28.820428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.820428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:28.820428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:28.820428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:28.820428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:28.820428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:28.820428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.820428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:28.820428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:28.820428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.820428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:28.820428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:28.820428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.820428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:28.820428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:28.820428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.820428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:17:28.820785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:17:28.820785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:28.820785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:28.820785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:28.820785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:28.820785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:28.820785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:28.820785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:28.820785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:28.820785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:28.820785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:28.820785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:28.820785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:28.820785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:28.820785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:28.820785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:28.820785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:28.820785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:28.820785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:28.821140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:28.821140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:28.821140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.821140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:28.827130 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:28.827317 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:28.827427 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:28.827574 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:28.827844 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:28.828004 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:28.828108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.828499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:28.828499 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:28.828499 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:28.828499 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:28.828499 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:28.828499 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:28.828499 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.828499 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:28.829348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.829348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:28.829348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:28.829348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:28.829348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.829348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:17:28.829348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:28.829348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:28.829348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:28.829348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:28.829348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:28.829348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:28.829615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:28.829615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:28.829615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:28.829615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:28.829615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:28.829615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:28.829615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:28.829615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:28.829615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:28.829615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:28.829615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:28.829615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:28.829615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:28.829892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.829892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:28.830433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.830433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:28.830433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:28.830433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:28.830433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:28.830433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:28.830433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.830433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:28.830433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:28.830433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.830433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:28.830433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:28.830433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.830433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:28.830433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:28.830433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:28.830790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:17:28.830790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:28.830790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:28.830790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:28.830790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:28.830790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:28.830790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:28.830790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:28.830790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:28.830790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:28.830790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:28.830790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:28.830790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:28.830790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:28.830790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:28.830790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:28.830790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:28.830790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:28.830790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:28.831155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:28.831155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:28.831155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:32.999213 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:32.999956 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:32.999956 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:33.001213 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:33.038817 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:33.039247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.041193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:33.041193 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:33.041193 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:33.041193 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:33.041193 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:33.041193 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:33.041193 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.041193 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:33.042224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.042224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:33.042224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:33.042224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:33.042224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.042224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:17:33.042224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:33.042224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:33.042224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:33.042224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:33.042224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:33.042224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:33.042817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:33.042817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:33.042817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:33.042817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:33.042817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:33.042817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:33.042817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:33.042817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:33.042817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:33.042817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:33.042817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:33.042817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:33.042817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:33.043380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.043380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:33.043880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.043880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:33.043880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:33.043880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:33.043880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:33.043880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:33.043880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.043880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:33.043880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:33.043880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.043880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:33.043880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:33.043880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.043880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:33.043880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:33.043880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.044743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:17:33.044743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:33.044743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:33.044743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:33.044743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:33.044743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:33.044743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:33.044743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:33.044743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:33.044743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:33.044743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:33.044743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:33.044743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:33.044743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:33.044743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:33.044743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:33.044743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:33.044743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:33.044743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:33.045532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:33.045532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.045532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:33.760219 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:33.760866 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:33.760993 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:33.762076 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:33.762444 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:33.762691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.763644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:33.763644 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:33.763644 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:33.763644 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:33.763644 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:33.763644 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:33.763644 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.763644 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:33.765650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.765650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:33.765650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:33.765650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:33.765650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.765650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:17:33.765650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:33.765650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:33.765650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:33.765650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:33.765650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:33.765650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:33.766183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:33.766183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:33.766183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:33.766183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:33.766183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:33.766183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:33.766183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:33.766183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:33.766183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:33.766183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:33.766183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:33.766183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:33.766183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:33.766571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.766571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:33.767040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.767040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:33.767040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:33.767040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:33.767040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:33.767040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:33.767040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.767040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:33.767040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:33.767040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.767040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:33.767040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:33.767040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.767040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:33.767040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:33.767040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.767490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:17:33.767490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:33.767490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:33.767490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:33.767490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:33.767490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:33.767490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:33.767490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:33.767490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:33.767490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:33.767490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:33.767490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:33.767490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:33.767490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:33.767490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:33.767490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:33.767490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:33.767490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:33.767490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:33.767986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:33.767986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.767986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:33.804716 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:33.805123 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:33.805295 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:33.805474 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:17:33.806138 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:33.806466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.807323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:33.807323 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:33.807323 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:33.807323 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:33.807323 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:33.807323 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:33.807323 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.807323 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:33.808462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.808462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:17:33.808462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:33.808462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:33.808462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:33.808462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:33.808462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:33.808462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:33.808462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:33.808462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:33.808462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:33.808462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:17:33.808462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:33.808462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:17:33.808462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:17:33.809112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:33.809112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:17:33.809112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:33.809112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:17:33.809112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:17:33.809112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:33.809112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:17:33.809112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.809112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:17:33.809809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.809809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:33.809809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:33.809809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:33.809809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.809809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:17:33.809809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:17:33.809809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:17:33.809809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:33.809809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:33.809809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:33.809809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:33.810239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:33.810239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:33.810239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:33.810239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:33.810239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:33.810239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:33.810239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:33.810239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:33.810239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:33.810239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:33.810239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:33.810239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:33.810239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:33.810239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:33.810761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:33.810761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:33.810761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.810761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:33.811371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.811371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:33.811371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:33.811371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:33.811371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:33.811371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:33.811371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.811371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:33.811371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:33.811371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.811371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:33.811371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:33.811371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.811371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:33.811371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:33.811371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.811981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:17:33.811981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:17:33.811981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:17:33.811981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:33.811981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:33.811981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:33.811981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:33.811981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:33.811981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:33.811981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:33.811981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:33.811981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:33.811981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:33.811981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:33.811981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:33.811981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:33.811981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:33.811981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:33.812621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:33.812621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:33.812621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:33.812621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:33.812621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:33.812621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.812621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:33.816427 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:33.816944 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:33.817127 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:33.817321 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:33.817869 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:33.818242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.818964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:33.818964 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:33.818964 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:33.818964 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:33.818964 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:33.818964 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:33.818964 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.818964 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:33.819713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.819713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:17:33.819713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:33.819713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:33.819713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:33.819713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:33.819713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:33.819713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:33.819713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:33.819713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:33.819713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:33.819713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:17:33.819713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:33.819713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:17:33.819713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:17:33.820312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:33.820312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:17:33.820312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:33.820312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:17:33.820312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:17:33.820312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:33.820312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:17:33.820312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.820312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:17:33.820697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.820697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:33.820697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:33.820697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:33.820697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.820697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:17:33.820697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:17:33.820697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:33.820697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:33.820697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:33.820697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:33.821127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:33.821127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:33.821127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:33.821127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:33.821127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:33.821127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:33.821127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:33.821127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:33.821127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:33.821127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:33.821127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:33.821127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:33.821127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:33.821127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:33.821733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:33.821733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:33.821733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.821733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:33.821733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.821733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:33.821733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:33.821733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:33.821733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:33.821733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:33.821733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.821733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:33.821733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:33.821733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.822244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:33.822244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:33.822244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.822244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:33.822244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:33.822244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.822244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:17:33.822244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:17:33.822244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:33.822244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:33.822244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:33.822244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:33.822244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:33.822244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:33.822244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:33.822244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:33.822244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:33.822244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:33.822244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:33.822932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:33.822932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:33.822932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:33.822932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:33.822932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:33.822932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:33.822932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:33.822932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:33.822932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:33.822932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.822932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:33.827523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:33.827803 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:33.827908 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:33.828051 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:33.828526 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:33.829006 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:33.829299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.830029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:33.830029 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:33.830029 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:33.830029 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:33.830029 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:33.830029 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:33.830029 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.830029 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:33.830839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.830839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:33.830839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:33.830839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:33.830839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.830839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:17:33.830839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:33.830839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:33.830839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:33.830839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:33.830839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:33.830839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:33.831320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:33.831320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:33.831320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:33.831320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:33.831320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:33.831320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:33.831320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:33.831320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:33.831320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:33.831320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:33.831320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:33.831320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:33.831320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:33.831838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.831838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:33.831838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.831838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:33.831838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:33.831838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:33.831838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:33.831838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:33.831838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.831838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:33.831838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:33.831838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.831838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:33.831838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:33.831838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.832433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:33.832433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:33.832433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:33.832433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:17:33.832433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:33.832433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:33.832433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:33.832433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:33.832433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:33.832433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:33.832433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:33.832433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:33.832433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:33.832433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:33.832433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:33.832433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:33.832433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:33.832433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:33.832433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:33.833029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:33.833029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:33.833029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:33.833029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:33.833029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:33.833029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:38.000063 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:38.001001 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:38.001212 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:38.002418 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:38.036990 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:38.037429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.038341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:38.038341 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:38.038341 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:38.038341 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:38.038341 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:38.038341 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:38.038341 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.038341 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:38.039170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.039170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:38.039170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:38.039170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:38.039170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.039170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:17:38.039170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:38.039170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:38.039170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:38.039170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:38.039170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:38.039170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:38.041157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:38.041157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:38.041157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:38.041157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:38.041157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:38.041157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:38.041157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:38.041157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:38.041157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:38.041157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:38.041157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:38.041157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:38.041157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:38.041851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.041851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:38.041851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.041851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:38.041851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:38.041851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:38.041851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:38.041851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:38.041851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.041851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:38.041851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:38.041851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.041851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:38.041851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:38.041851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.042579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:38.042579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:38.042579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.042579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:17:38.042579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:38.042579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:38.042579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:38.042579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:38.042579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:38.042579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:38.042579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:38.042579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:38.042579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:38.042579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:38.042579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:38.042579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:38.042579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:38.042579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:38.042579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:38.044311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:38.044311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:38.044311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:38.044311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:38.044311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.044311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:38.761185 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:38.762069 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:38.762283 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:38.763449 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:38.763732 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:38.763996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.764877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:38.764877 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:38.764877 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:38.764877 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:38.764877 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:38.764877 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:38.764877 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.764877 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:38.766463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.766463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:38.766463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:38.766463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:38.766463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.766463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:17:38.766463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:38.766463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:38.766463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:38.766463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:38.766463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:38.766463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:38.767101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:38.767101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:38.767101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:38.767101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:38.767101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:38.767101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:38.767101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:38.767101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:38.767101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:38.767101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:38.767101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:38.767101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:38.767101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:38.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:38.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:38.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:38.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:38.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:38.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:38.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:38.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:38.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:38.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:38.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.768455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:38.768455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:38.768455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.768455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:17:38.768455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:38.768455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:38.768455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:38.768455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:38.768455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:38.768455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:38.768455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:38.768455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:38.768455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:38.768455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:38.768455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:38.768455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:38.768455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:38.768455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:38.768455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:38.769344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:38.769344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:38.769344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:38.769344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:38.769344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.769344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:38.805246 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:38.805813 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:38.805927 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:38.806109 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:17:38.806723 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:38.806961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.807721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:38.807721 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:38.807721 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:38.807721 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:38.807721 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:38.807721 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:38.807721 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.807721 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:38.808557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.808557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:17:38.808557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:38.808557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:38.808557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:38.808557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:38.808557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:38.808557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:38.808557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:38.808557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:38.808557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:38.808557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:17:38.808557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:38.808557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:17:38.808557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:17:38.809004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:38.809004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:17:38.809004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:38.809004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:17:38.809004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:17:38.809004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:38.809004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:17:38.809004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.809004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:17:38.809467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.809467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:38.809467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:38.809467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:38.809467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.809467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:17:38.809467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:17:38.809467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:17:38.809467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:38.809467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:38.809467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:38.809467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:38.810550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:38.810550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:38.810550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:38.810550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:38.810550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:38.810550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:38.810550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:38.810550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:38.810550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:38.810550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:38.810550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:38.810550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:38.810550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:38.810550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:38.810934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:38.810934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:38.810934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.810934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:38.810934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.810934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:38.810934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:38.810934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:38.810934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:38.810934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:38.810934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.810934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:38.810934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:38.810934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.811321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:38.811321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:38.811321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.811321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:38.811321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:38.811321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.811321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:17:38.811321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:17:38.811321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:17:38.811321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:38.811321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:38.811321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:38.811321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:38.811321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:38.811321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:38.811321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:38.811321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:38.811321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:38.811321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:38.812206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:38.812206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:38.812206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:38.812206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:38.812206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:38.812206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:38.812206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:38.812206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:38.812206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:38.812206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:38.812206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.812206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:38.816837 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:38.817122 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:38.817244 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:38.817381 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:38.818092 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:38.818567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.819355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:38.819355 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:38.819355 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:38.819355 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:38.819355 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:38.819355 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:38.819355 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.819355 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:38.820135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.820135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:17:38.820135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:38.820135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:38.820135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:38.820135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:38.820135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:38.820135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:38.820135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:38.820135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:38.820135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:38.820135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:17:38.820135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:38.820135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:17:38.820135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:17:38.820865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:38.820865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:17:38.820865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:38.820865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:17:38.820865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:17:38.820865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:38.820865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:17:38.820865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.820865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:17:38.821269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.821269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:38.821269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:38.821269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:38.821269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.821269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:17:38.821269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:17:38.821269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:38.821269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:38.821269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:38.821269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:38.821850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:38.821850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:38.821850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:38.821850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:38.821850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:38.821850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:38.821850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:38.821850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:38.821850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:38.821850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:38.821850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:38.821850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:38.821850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:38.821850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:38.822492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:38.822492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:38.822492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.822492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:38.822729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.822729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:38.822729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:38.822729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:38.822729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:38.822729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:38.822729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.822729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:38.822729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:38.822729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.822729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:38.822729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:38.822729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.822729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:38.822729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:38.822729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.822729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:17:38.823495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:17:38.823495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:38.823495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:38.823495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:38.823495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:38.823495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:38.823495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:38.823495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:38.823495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:38.823495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:38.823495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:38.823495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:38.823495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:38.823495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:38.823495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:38.823495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:38.823495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:38.823495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:38.823495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:38.824452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:38.824452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:38.824452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.824452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:38.827854 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:38.828154 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:38.828270 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:38.828425 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:38.828910 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:38.829284 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:38.829525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.830500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:38.830500 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:38.830500 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:38.830500 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:38.830500 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:38.830500 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:38.830500 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.830500 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:38.832032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.832032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:38.832032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:38.832032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:38.832032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.832032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:17:38.832032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:38.832032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:38.832032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:38.832032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:38.832032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:38.832032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:38.832474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:38.832474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:38.832474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:38.832474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:38.832474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:38.832474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:38.832474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:38.832474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:38.832474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:38.832474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:38.832474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:38.832474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:38.832474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:38.832925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.832925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:38.833977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.833977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:38.833977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:38.833977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:38.833977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:38.833977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:38.833977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.833977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:38.833977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:38.833977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.833977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:38.833977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:38.833977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.833977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:38.833977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:38.833977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:38.834537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:17:38.834537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:38.834537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:38.834537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:38.834537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:38.834537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:38.834537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:38.834537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:38.834537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:38.834537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:38.834537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:38.834537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:38.834537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:38.834537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:38.834537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:38.834537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:38.834537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:38.834537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:38.834537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:38.835136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:38.835136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:38.835136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:43.001128 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:43.001914 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:43.002154 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:43.003579 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:43.037095 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:43.037296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.037848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:43.037848 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:43.037848 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:43.037848 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:43.037848 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:43.037848 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:43.037848 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.037848 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:43.038653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.038653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:43.038653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:43.038653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:43.038653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.038653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:17:43.038653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:43.038653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:43.038653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:43.038653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:43.038653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:43.038653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:43.039296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:43.039296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:43.039296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:43.039296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:43.039296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:43.039296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:43.039296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:43.039296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:43.039296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:43.039296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:43.039296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:43.039296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:43.039296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:43.039922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.039922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:43.039922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.039922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:43.039922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:43.039922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:43.039922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:43.039922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:43.039922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.039922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:43.039922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:43.039922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.039922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:43.039922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:43.039922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.040779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:43.040779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:43.040779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.040779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:17:43.040779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:43.040779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:43.040779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:43.040779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:43.040779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:43.040779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:43.040779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:43.040779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:43.040779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:43.040779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:43.040779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:43.040779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:43.040779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:43.040779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:43.040779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:43.041612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:43.041612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:43.041612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:43.041612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:43.041612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.041612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:43.762072 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:43.762831 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:43.763063 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:43.764104 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:43.764398 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:43.764630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.765897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:43.765897 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:43.765897 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:43.765897 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:43.765897 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:43.765897 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:43.765897 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.765897 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:43.767413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.767413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:43.767413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:43.767413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:43.767413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.767413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:17:43.767413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:43.767413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:43.767413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:43.767413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:43.767413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:43.767413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:43.767984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:43.767984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:43.767984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:43.767984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:43.767984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:43.767984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:43.767984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:43.767984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:43.767984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:43.767984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:43.767984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:43.767984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:43.767984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:43.768565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.768565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:43.769273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.769273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:43.769273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:43.769273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:43.769273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:43.769273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:43.769273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.769273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:43.769273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:43.769273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.769273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:43.769273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:43.769273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.769273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:43.769273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:43.769273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.770023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:17:43.770023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:43.770023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:43.770023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:43.770023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:43.770023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:43.770023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:43.770023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:43.770023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:43.770023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:43.770023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:43.770023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:43.770023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:43.770023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:43.770023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:43.770023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:43.770023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:43.770023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:43.770023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:43.770830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:43.770830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.770830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:43.806424 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:43.807005 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:43.807114 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:43.807218 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:17:43.807617 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:43.807837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.808288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:43.808288 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:43.808288 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:43.808288 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:43.808288 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:43.808288 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:43.808288 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.808288 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:43.809138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.809138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:17:43.809138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:43.809138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:43.809138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:43.809138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:43.809138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:43.809138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:43.809138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:43.809138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:43.809138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:43.809138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:17:43.809138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:43.809138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:17:43.809138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:17:43.809861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:43.809861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:17:43.809861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:43.809861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:17:43.809861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:17:43.809861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:43.809861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:17:43.809861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.809861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:17:43.809861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.809861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:43.809861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:43.809861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:43.809861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.810481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:17:43.810481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:17:43.810481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:17:43.810481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:43.810481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:43.810481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:43.810481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:43.810481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:43.810481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:43.810481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:43.810481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:43.810481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:43.810481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:43.811097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:43.811097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:43.811097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:43.811097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:43.811097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:43.811097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:43.811097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:43.811097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:43.811097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:43.811097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:43.811097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.811097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:43.811647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.811647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:43.811647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:43.811647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:43.811647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:43.811647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:43.811647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.811647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:43.811647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:43.811647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.811647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:43.811647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:43.811647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.811647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:43.811647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:43.811647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.812647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:17:43.812647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:17:43.812647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:17:43.812647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:43.812647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:43.812647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:43.812647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:43.812647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:43.812647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:43.812647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:43.812647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:43.812647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:43.812647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:43.812647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:43.812647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:43.812647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:43.812647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:43.812647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:43.813471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:43.813471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:43.813471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:43.813471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:43.813471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:43.813471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.813471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:43.817316 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:43.817696 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:43.817832 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:43.817975 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:43.818312 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:43.818541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.819026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:43.819026 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:43.819026 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:43.819026 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:43.819026 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:43.819026 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:43.819026 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.819026 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:43.819734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.819734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:17:43.819734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:43.819734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:43.819734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:43.819734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:43.819734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:43.819734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:43.819734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:43.819734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:43.819734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:43.819734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:17:43.819734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:43.819734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:17:43.819734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:17:43.820258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:43.820258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:17:43.820258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:43.820258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:17:43.820258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:17:43.820258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:43.820258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:17:43.820258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.820258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:17:43.820778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.820778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:43.820778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:43.820778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:43.820778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.820778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:17:43.820778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:17:43.820778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:43.820778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:43.820778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:43.820778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:43.821171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:43.821171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:43.821171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:43.821171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:43.821171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:43.821171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:43.821171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:43.821171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:43.821171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:43.821171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:43.821171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:43.821171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:43.821171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:43.821171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:43.821552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:43.821552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:43.821552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.821552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:43.821955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.821955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:43.821955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:43.821955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:43.821955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:43.821955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:43.821955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.821955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:43.821955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:43.821955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.821955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:43.821955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:43.821955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.821955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:43.821955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:43.821955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.821955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:17:43.822444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:17:43.822444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:43.822444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:43.822444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:43.822444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:43.822444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:43.822444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:43.822444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:43.822444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:43.822444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:43.822444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:43.822444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:43.822444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:43.822444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:43.822444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:43.822444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:43.822444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:43.822444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:43.822444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:43.822943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:43.822943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:43.822943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.822943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:43.828343 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:43.828723 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:43.828886 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:43.829043 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:43.829278 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:43.829486 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:43.829629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.830210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:43.830210 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:43.830210 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:43.830210 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:43.830210 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:43.830210 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:43.830210 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.830210 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:43.831255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.831255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:43.831255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:43.831255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:43.831255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.831255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:17:43.831255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:43.831255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:43.831255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:43.831255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:43.831255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:43.831255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:43.831589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:43.831589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:43.831589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:43.831589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:43.831589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:43.831589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:43.831589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:43.831589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:43.831589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:43.831589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:43.831589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:43.831589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:43.831589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:43.831964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.831964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:43.832967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.832967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:43.832967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:43.832967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:43.832967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:43.832967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:43.832967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.832967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:43.832967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:43.832967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.832967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:43.832967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:43.832967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.832967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:43.832967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:43.832967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:43.833375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:17:43.833375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:43.833375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:43.833375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:43.833375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:43.833375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:43.833375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:43.833375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:43.833375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:43.833375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:43.833375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:43.833375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:43.833375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:43.833375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:43.833375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:43.833375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:43.833375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:43.833375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:43.833375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:43.833906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:43.833906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:43.833906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:48.002192 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:48.003077 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:48.003170 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:48.004519 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:48.039530 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:48.039726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.040189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:48.040189 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:48.040189 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:48.040189 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:48.040189 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:48.040189 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:48.040189 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.040189 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:48.041119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.041119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:48.041119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:48.041119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:48.041119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.041119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:17:48.041119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:48.041119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:48.041119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:48.041119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:48.041119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:48.041119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:48.041740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:48.041740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:48.041740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:48.041740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:48.041740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:48.041740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:48.041740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:48.041740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:48.041740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:48.041740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:48.041740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:48.041740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:48.041740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:48.042343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.042343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:48.042343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.042343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:48.042343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:48.042343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:48.042343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:48.042343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:48.042343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.042343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:48.042343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:48.042343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.042343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:48.042343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:48.042343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.043150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:48.043150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:48.043150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.043150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:17:48.043150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:48.043150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:48.043150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:48.043150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:48.043150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:48.043150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:48.043150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:48.043150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:48.043150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:48.043150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:48.043150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:48.043150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:48.043150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:48.043150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:48.043150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:48.044155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:48.044155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:48.044155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:48.044155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:48.044155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.044155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:48.762928 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:48.763540 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:48.763765 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:48.764738 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:48.765047 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:48.765330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.766197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:48.766197 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:48.766197 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:48.766197 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:48.766197 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:48.766197 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:48.766197 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.766197 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:48.767593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.767593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:48.767593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:48.767593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:48.767593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.767593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:17:48.767593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:48.767593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:48.767593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:48.767593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:48.767593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:48.767593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:48.767987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:48.767987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:48.767987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:48.767987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:48.767987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:48.767987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:48.767987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:48.767987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:48.767987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:48.767987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:48.767987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:48.767987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:48.767987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:48.768328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.768328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:48.769312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.769312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:48.769312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:48.769312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:48.769312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:48.769312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:48.769312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.769312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:48.769312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:48.769312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.769312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:48.769312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:48.769312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.769312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:48.769312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:48.769312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.769806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:17:48.769806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:48.769806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:48.769806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:48.769806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:48.769806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:48.769806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:48.769806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:48.769806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:48.769806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:48.769806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:48.769806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:48.769806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:48.769806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:48.769806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:48.769806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:48.769806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:48.769806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:48.769806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:48.770311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:48.770311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.770311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:48.807020 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:48.807480 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:48.807700 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:48.807907 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:17:48.808402 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:48.808771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.809604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:48.809604 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:48.809604 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:48.809604 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:48.809604 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:48.809604 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:48.809604 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.809604 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:48.810535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.810535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:17:48.810535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:48.810535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:48.810535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:48.810535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:48.810535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:48.810535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:48.810535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:48.810535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:48.810535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:48.810535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:17:48.810535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:48.810535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:17:48.810535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:17:48.811338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:48.811338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:17:48.811338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:48.811338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:17:48.811338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:17:48.811338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:48.811338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:17:48.811338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.811338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:17:48.811800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.811800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:48.811800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:48.811800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:48.811800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.811800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:17:48.811800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:17:48.811800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:17:48.811800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:48.811800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:48.811800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:48.811800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:48.812360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:48.812360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:48.812360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:48.812360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:48.812360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:48.812360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:48.812360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:48.812360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:48.812360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:48.812360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:48.812360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:48.812360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:48.812360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:48.812360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:48.813016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:48.813016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:48.813016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.813016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:48.813016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.813016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:48.813016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:48.813016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:48.813016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:48.813016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:48.813016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.813016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:48.813016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:48.813016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.813671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:48.813671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:48.813671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.813671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:48.813671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:48.813671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.813671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:17:48.813671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:17:48.813671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:17:48.813671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:48.813671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:48.813671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:48.813671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:48.813671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:48.813671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:48.813671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:48.813671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:48.813671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:48.813671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:48.814666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:48.814666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:48.814666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:48.814666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:48.814666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:48.814666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:48.814666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:48.814666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:48.814666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:48.814666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:48.814666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.814666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:48.817929 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:48.818194 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:48.818310 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:48.818454 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:48.818695 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:48.818956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.819389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:48.819389 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:48.819389 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:48.819389 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:48.819389 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:48.819389 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:48.819389 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.819389 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:48.820125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.820125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:17:48.820125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:48.820125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:48.820125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:48.820125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:48.820125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:48.820125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:48.820125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:48.820125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:48.820125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:48.820125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:17:48.820125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:48.820125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:17:48.820125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:17:48.820653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:48.820653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:17:48.820653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:48.820653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:17:48.820653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:17:48.820653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:48.820653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:17:48.820653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.820653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:17:48.821049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.821049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:48.821049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:48.821049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:48.821049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.821049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:17:48.821049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:17:48.821049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:48.821049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:48.821049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:48.821049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:48.821413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:48.821413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:48.821413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:48.821413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:48.821413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:48.821413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:48.821413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:48.821413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:48.821413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:48.821413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:48.821413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:48.821413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:48.821413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:48.821413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:48.821795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:48.821795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:48.821795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.821795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:48.822168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.822168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:48.822168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:48.822168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:48.822168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:48.822168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:48.822168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.822168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:48.822168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:48.822168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.822168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:48.822168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:48.822168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.822168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:48.822168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:48.822168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.822168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:17:48.822695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:17:48.822695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:48.822695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:48.822695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:48.822695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:48.822695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:48.822695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:48.822695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:48.822695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:48.822695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:48.822695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:48.822695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:48.822695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:48.822695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:48.822695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:48.822695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:48.822695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:48.822695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:48.822695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:48.823162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:48.823162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:48.823162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.823162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:48.828847 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:48.829105 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:48.829221 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:48.829391 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:48.829646 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:48.829894 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:48.830029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.830485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:48.830485 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:48.830485 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:48.830485 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:48.830485 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:48.830485 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:48.830485 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.830485 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:48.831277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.831277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:48.831277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:48.831277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:48.831277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.831277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:17:48.831277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:48.831277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:48.831277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:48.831277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:48.831277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:48.831277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:48.831630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:48.831630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:48.831630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:48.831630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:48.831630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:48.831630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:48.831630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:48.831630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:48.831630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:48.831630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:48.831630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:48.831630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:48.831630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:48.831967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.831967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:48.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:48.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:48.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:48.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:48.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:48.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:48.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:48.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:48.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:48.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:48.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:48.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:48.832766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:17:48.832766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:48.832766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:48.832766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:48.832766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:48.832766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:48.832766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:48.832766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:48.832766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:48.832766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:48.832766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:48.832766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:48.832766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:48.832766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:48.832766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:48.832766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:48.832766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:48.832766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:48.832766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:48.833216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:48.833216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:48.833216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:53.002917 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:53.003733 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:53.003808 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:53.005273 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:53.042847 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:53.042847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.043280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:53.043280 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:53.043280 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:53.043280 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:53.043280 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:53.043280 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:53.043280 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.043280 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:53.044055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.044055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:53.044055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:53.044055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:53.044055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.044055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:17:53.044055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:53.044055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:53.044055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:53.044055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:53.044055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:53.044055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:53.044700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:53.044700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:53.044700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:53.044700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:53.044700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:53.044700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:53.044700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:53.044700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:53.044700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:53.044700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:53.044700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:53.044700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:53.044700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:53.045343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.045343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:53.045343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.045343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:53.045343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:53.045343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:53.045343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:53.045343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:53.045343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.045343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:53.045343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:53.045343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.045343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:53.045343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:53.045343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.046029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:53.046029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:53.046029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.046029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:17:53.046029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:53.046029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:53.046029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:53.046029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:53.046029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:53.046029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:53.046029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:53.046029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:53.046029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:53.046029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:53.046029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:53.046029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:53.046029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:53.046029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:53.046029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:53.046807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:53.046807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:53.046807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:53.046807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:53.046807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.046807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:53.763553 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:53.764422 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:53.764614 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:53.768101 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:53.768101 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:53.768101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.768101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:53.768101 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:53.768101 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:53.768101 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:53.768101 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:53.768101 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:53.768101 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.768101 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:53.769371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.769371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:53.769371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:53.769371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:53.769371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.769371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:17:53.769371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:53.769371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:53.769371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:53.769371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:53.769371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:53.769371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:53.770077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:53.770077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:53.770077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:53.770077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:53.770077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:53.770077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:53.770077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:53.770077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:53.770077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:53.770077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:53.770077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:53.770077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:53.770077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:53.770600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.770600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:53.774370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.774370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:53.774370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:53.774370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:53.774370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:53.774370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:53.774370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.774370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:53.774370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:53.774370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.774370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:53.774370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:53.774370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.774370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:53.774370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:53.774370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.775286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:17:53.775286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:53.775286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:53.775286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:53.775286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:53.775286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:53.775286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:53.775286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:53.775286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:53.775286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:53.775286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:53.775286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:53.775286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:53.775286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:53.775286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:53.775286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:53.775286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:53.775286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:53.775286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:53.775827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:53.775827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.775827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:53.807644 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:53.808400 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:53.808612 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:53.811099 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:17:53.811695 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:53.811953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.812601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:53.812601 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:53.812601 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:53.812601 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:53.812601 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:53.812601 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:53.812601 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.812601 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:53.813501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.813501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:17:53.813501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:53.813501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:53.813501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:53.813501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:53.813501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:53.813501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:53.813501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:53.813501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:53.813501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:53.813501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:17:53.813501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:53.813501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:17:53.813501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:17:53.814020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:53.814020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:17:53.814020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:53.814020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:17:53.814020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:17:53.814020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:53.814020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:17:53.814020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.814020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:17:53.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:53.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:53.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:53.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:17:53.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:17:53.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:17:53.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:53.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:53.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:53.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:53.814849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:53.814849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:53.814849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:53.814849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:53.814849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:53.814849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:53.814849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:53.814849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:53.814849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:53.814849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:53.814849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:53.814849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:53.814849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:53.814849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:53.815256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:53.815256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:53.815256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.815256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:53.815436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.815436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:53.815436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:53.815436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:53.815436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:53.815436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:53.815436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.815436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:53.815436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:53.815436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.815436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:53.815436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:53.815436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.815436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:53.815436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:53.815436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.816160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:17:53.816160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:17:53.816160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:17:53.816160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:53.816160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:53.816160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:53.816160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:53.816160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:53.816160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:53.816160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:53.816160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:53.816160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:53.816160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:53.816160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:53.816160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:53.816160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:53.816160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:53.816160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:53.816635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:53.816635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:53.816635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:53.816635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:53.816635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:53.816635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.816635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:53.818260 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:53.818541 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:53.818659 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:53.818892 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:53.819231 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:53.819479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.820102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:53.820102 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:53.820102 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:53.820102 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:53.820102 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:53.820102 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:53.820102 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.820102 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:53.820876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.820876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:17:53.820876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:53.820876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:53.820876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:53.820876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:53.820876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:53.820876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:53.820876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:53.820876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:53.820876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:53.820876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:17:53.820876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:53.820876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:17:53.820876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:17:53.821265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:53.821265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:17:53.821265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:53.821265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:17:53.821265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:17:53.821265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:53.821265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:17:53.821265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.821265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:17:53.821726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.821726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:53.821726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:53.821726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:53.821726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.821726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:17:53.821726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:17:53.821726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:53.821726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:53.821726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:53.821726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:53.822069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:53.822069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:53.822069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:53.822069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:53.822069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:53.822069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:53.822069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:53.822069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:53.822069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:53.822069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:53.822069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:53.822069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:53.822069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:53.822069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:53.822458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:53.822458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:53.822458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.822458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:53.823626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.823626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:53.823626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:53.823626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:53.823626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:53.823626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:53.823626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.823626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:53.823626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:53.823626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.823626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:53.823626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:53.823626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.823626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:53.823626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:53.823626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.823626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:17:53.825101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:17:53.825101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:53.825101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:53.825101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:53.825101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:53.825101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:53.825101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:53.825101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:53.825101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:53.825101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:53.825101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:53.825101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:53.825101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:53.825101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:53.825101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:53.825101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:53.825101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:53.825101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:53.825101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:53.826024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:53.826024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:53.826024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.826024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:53.829262 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:53.829578 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:53.829716 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:53.829874 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:53.830380 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:53.830908 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:53.831272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.832132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:53.832132 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:53.832132 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:53.832132 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:53.832132 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:53.832132 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:53.832132 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.832132 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:53.833000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.833000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:53.833000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:53.833000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:53.833000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.833000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:17:53.833000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:53.833000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:53.833000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:53.833000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:53.833000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:53.833000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:53.833596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:53.833596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:53.833596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:53.833596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:53.833596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:53.833596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:53.833596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:53.833596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:53.833596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:53.833596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:53.833596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:53.833596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:53.833596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:53.834314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.834314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:53.834314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.834314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:53.834314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:53.834314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:53.834314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:53.834314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:53.834314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.834314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:53.834314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:53.834314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.834314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:53.834314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:53.834314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:53.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:53.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:53.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:17:53.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:53.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:53.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:53.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:53.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:53.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:53.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:53.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:53.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:53.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:53.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:53.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:53.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:53.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:53.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:53.836575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:53.836575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:53.836575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:53.836575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:53.836575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:53.836575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:58.003427 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:58.004068 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:58.004262 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:58.005526 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:58.044445 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:58.044445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.044635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:58.044635 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:58.044635 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:58.044635 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:58.044635 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:58.044635 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:58.044635 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.044635 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:58.045417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.045417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:58.045417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:58.045417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:58.045417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.045417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:17:58.045417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:58.045417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:58.045417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:58.045417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:58.045417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:58.045417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:58.045772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:58.045772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:58.045772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:58.045772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:58.045772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:58.045772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:58.045772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:58.045772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:58.045772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:58.045772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:58.045772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:58.045772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:58.045772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:58.046131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.046131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:58.046213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.046213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:58.046213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:58.046213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:58.046213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:58.046213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:58.046213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.046213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:58.046213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:58.046213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.046213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:58.046213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:58.046213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.046213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:58.046213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:58.046213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:17:58.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:58.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:58.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:58.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:58.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:58.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:58.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:58.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:58.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:58.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:58.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:58.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:58.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:58.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:58.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:58.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:58.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:58.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:58.047157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:58.047157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.047157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:58.764259 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:58.765296 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:58.765296 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:58.766372 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:58.767807 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:58.767807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.768274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:58.768274 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:58.768274 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:58.768274 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:58.768274 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:58.768274 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:58.768274 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.768274 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:58.770128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.770128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:58.770128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:58.770128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:58.770128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.770128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:17:58.770128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:58.770128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:58.770128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:58.770128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:58.770128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:58.770128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:58.770493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:58.770493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:58.770493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:58.770493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:58.770493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:58.770493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:58.770493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:58.770493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:58.770493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:58.770493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:58.770493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:58.770493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:58.770493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:58.770791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.770791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:58.771839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.771839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:58.771839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:58.771839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:58.771839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:58.771839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:58.771839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.771839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:58.771839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:58.771839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.771839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:58.771839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:58.771839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.771839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:58.771839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:58.771839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.772243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:17:58.772243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:58.772243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:58.772243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:58.772243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:58.772243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:58.772243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:58.772243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:58.772243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:58.772243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:58.772243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:58.772243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:58.772243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:58.772243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:58.772243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:58.772243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:58.772243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:58.772243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:58.772243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:58.772686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:58.772686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.772686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:58.808440 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:58.808981 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:58.809122 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:58.809320 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:17:58.810109 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:58.810469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.811291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:58.811291 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:58.811291 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:58.811291 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:58.811291 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:58.811291 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:58.811291 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.811291 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:58.812536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.812536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:17:58.812536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:58.812536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:58.812536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:58.812536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:58.812536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:58.812536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:58.812536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:58.812536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:58.812536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:58.812536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:17:58.812536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:58.812536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:17:58.812536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:17:58.813389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:58.813389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:17:58.813389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:58.813389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:17:58.813389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:17:58.813389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:58.813389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:17:58.813389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.813389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:17:58.813984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.813984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:58.813984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:58.813984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:58.813984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.813984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:17:58.813984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:17:58.813984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:17:58.813984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:58.813984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:58.813984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:58.813984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:58.814617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:58.814617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:58.814617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:58.814617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:58.814617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:58.814617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:58.814617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:58.814617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:58.814617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:58.814617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:58.814617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:58.814617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:58.814617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:58.814617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:58.815389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:58.815389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:58.815389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.815389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:58.815991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.815991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:58.815991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:58.815991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:58.815991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:58.815991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:58.815991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.815991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:58.815991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:58.815991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.815991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:58.815991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:58.815991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.815991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:58.815991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:58.815991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.817332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:17:58.817332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:17:58.817332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:17:58.817332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:58.817332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:58.817332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:58.817332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:58.817332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:58.817332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:58.817332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:58.817332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:58.817332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:58.817332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:58.817332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:58.817332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:58.817332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:58.817332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:58.817332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:58.818078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:58.818078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:58.818078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:58.818078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:58.818078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:58.818078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.818078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:58.819168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:58.819558 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:58.819558 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:58.819885 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:58.820456 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:58.820696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.821337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:58.821337 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:58.821337 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:58.821337 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:58.821337 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:58.821337 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:58.821337 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.821337 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:58.822105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.822105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:17:58.822105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:58.822105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:58.822105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:58.822105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:58.822105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:58.822105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:58.822105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:58.822105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:17:58.822105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:58.822105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:17:58.822105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:58.822105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:17:58.822105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:17:58.822914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:58.822914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:17:58.822914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:58.822914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:17:58.822914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:17:58.822914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:58.822914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:17:58.822914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.822914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:17:58.823390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.823390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:58.823390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:58.823390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:58.823390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.823390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:17:58.823390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:17:58.823390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:58.823390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:58.823390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:58.823390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:58.824592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:58.824592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:58.824592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:58.824592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:58.824592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:58.824592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:58.824592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:58.824592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:58.824592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:58.824592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:58.824592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:58.824592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:58.824592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:58.824592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:58.825057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:58.825057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:58.825057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.825057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:58.825057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.825057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:58.825057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:58.825057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:58.825057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:58.825057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:58.825057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.825057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:58.825057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:58.825057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.825897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:58.825897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:58.825897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.825897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:58.825897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:58.825897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.825897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:17:58.825897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:17:58.825897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:17:58.825897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:17:58.825897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:58.825897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:58.825897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:58.825897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:58.825897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:58.825897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:58.825897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:58.825897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:58.825897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:58.826812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:58.826812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:58.826812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:58.826812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:58.826812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:58.826812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:58.826812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:58.826812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:58.826812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:58.826812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.826812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:17:58.829722 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:17:58.829995 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:17:58.830122 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:17:58.830263 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:17:58.830568 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:58.830785 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:17:58.830907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.831433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:17:58.831433 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:58.831433 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:17:58.831433 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:17:58.831433 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:58.831433 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:17:58.831433 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.831433 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:17:58.832476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.832476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:17:58.832476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:58.832476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:17:58.832476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.832476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:17:58.832476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:58.832476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:58.832476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:58.832476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:58.832476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:58.832476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:58.832919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:17:58.832919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:58.832919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:17:58.832919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:58.832919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:17:58.832919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:17:58.832919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:58.832919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:17:58.832919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:58.832919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:17:58.832919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:17:58.832919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:58.832919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:17:58.833295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.833295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:17:58.833455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.833455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:17:58.833455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:17:58.833455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:17:58.833455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:17:58.833455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:17:58.833455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.833455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:17:58.833455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:17:58.833455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.833455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:17:58.833455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:17:58.833455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.833455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:17:58.833455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:17:58.833455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:17:58.833940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:17:58.833940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:17:58.833940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:58.833940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:58.833940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:17:58.833940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:17:58.833940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:17:58.833940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:17:58.833940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:17:58.833940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:17:58.833940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:17:58.833940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:17:58.833940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:17:58.833940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:17:58.833940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:17:58.833940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:17:58.833940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:17:58.833940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:17:58.833940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:17:58.834504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:17:58.834504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:17:58.834504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:03.004423 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:03.005521 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:03.005755 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:03.007039 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:03.015249 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=241 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:18:03.040544 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:03.040960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.041963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:03.041963 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:03.041963 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:03.041963 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:03.041963 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:03.041963 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:03.041963 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.041963 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:03.042933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.042933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:03.042933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:03.042933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:03.042933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.042933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:18:03.042933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:03.042933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:03.042933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:03.042933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:03.042933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:03.042933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:03.043261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:03.043261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:03.043261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:03.043261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:03.043261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:03.043261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:03.043261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:03.043261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:03.043261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:03.043261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:03.043261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:03.043261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:03.043261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:03.043592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.043592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:03.043789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.043789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:03.043789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:03.043789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:03.043789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:03.043789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:03.043789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.043789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:03.043789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:03.043789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.043789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:03.043789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:03.043789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.043789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:03.043789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:03.043789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.044938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:18:03.044938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:03.044938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:03.044938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:03.044938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:03.044938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:03.044938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:03.044938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:03.044938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:03.044938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:03.044938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:03.044938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:03.044938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:03.044938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:03.044938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:03.044938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:03.044938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:03.044938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:03.044938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:03.045483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:03.045483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.045483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:03.764938 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:03.765351 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:03.765712 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:03.766708 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:03.767146 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:03.767418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.768367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:03.768367 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:03.768367 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:03.768367 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:03.768367 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:03.768367 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:03.768367 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.768367 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:03.769937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.769937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:03.769937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:03.769937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:03.769937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.769937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:18:03.769937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:03.769937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:03.769937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:03.769937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:03.769937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:03.769937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:03.770432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:03.770432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:03.770432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:03.770432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:03.770432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:03.770432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:03.770432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:03.770432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:03.770432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:03.770432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:03.770432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:03.770432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:03.770432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:03.770967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.770967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:03.771633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.771633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:03.771633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:03.771633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:03.771633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:03.771633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:03.771633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.771633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:03.771633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:03.771633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.771633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:03.771633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:03.771633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.771633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:03.771633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:03.771633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.772987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:18:03.772987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:03.772987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:03.772987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:03.772987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:03.772987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:03.772987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:03.772987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:03.772987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:03.772987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:03.772987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:03.772987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:03.772987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:03.772987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:03.772987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:03.772987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:03.772987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:03.772987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:03.772987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:03.773758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:03.773758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.773758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:03.808965 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:03.809441 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:03.809636 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:03.809843 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:18:03.810473 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:03.810918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.811899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:03.811899 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:03.811899 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:03.811899 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:03.811899 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:03.811899 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:03.811899 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.811899 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:03.813119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.813119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:18:03.813119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:03.813119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:03.813119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:03.813119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:03.813119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:03.813119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:03.813119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:03.813119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:03.813119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:03.813119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:18:03.813119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:03.813119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:18:03.813119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:18:03.815382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:03.815382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:18:03.815382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:03.815382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:18:03.815382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:18:03.815382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:03.815382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:18:03.815382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.815382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:18:03.815382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.815382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:03.815382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:03.815382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:03.815382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.816055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:18:03.816055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:18:03.816055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:18:03.816055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:03.816055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:03.816055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:03.816055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:03.816055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:03.816055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:03.816055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:03.816055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:03.816055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:03.816055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:03.816648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:03.816648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:03.816648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:03.816648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:03.816648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:03.816648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:03.816648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:03.816648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:03.816648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:03.816648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:03.816648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.816648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:03.818877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.818877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:03.818877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:03.818877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:03.818877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:03.818877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:03.818877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.818877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:03.818877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:03.818877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.818877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:03.818877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:03.818877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.818877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:03.818877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:03.818877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.820359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:18:03.820359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:18:03.820359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:18:03.820359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:03.820359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:03.820359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:03.820359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:03.820359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:03.820359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:03.820359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:03.820359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:03.820359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:03.820359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:03.820359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:03.820359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:03.820359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:03.820359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:03.820359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:03.822141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:03.822141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:03.822141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:03.822141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:03.822141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:03.822141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.822141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:03.822141 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:03.822141 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:03.822141 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:03.822710 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:03.822710 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:03.822710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.822710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:03.822710 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:03.822710 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:03.822710 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:03.822710 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:03.822710 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:03.822710 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.822710 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:03.822710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.823511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:18:03.823511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:03.823511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:03.823511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:03.823511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:03.823511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:03.823511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:03.823511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:03.823511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:03.823511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:03.823511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:18:03.823511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:03.823511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:18:03.823511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:18:03.823511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:03.823511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:18:03.823511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:03.823946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:18:03.823946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:18:03.823946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:03.823946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:18:03.823946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.823946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:18:03.823946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.823946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:03.823946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:03.823946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:03.823946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.823946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:18:03.823946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:18:03.824263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:03.824263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:03.824263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:03.824263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:03.824263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:03.824263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:03.824263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:03.824263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:03.824263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:03.824263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:03.824263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:03.824263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:03.824263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:03.824611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:03.824611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:03.824611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:03.824611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:03.824611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:03.824611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:03.824611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:03.824611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.824611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:03.824850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.824850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:03.824850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:03.824850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:03.824850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:03.824850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:03.824850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.824850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:03.824850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:03.824850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.824850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:03.824850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:03.824850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.824850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:03.824850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:03.824850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.824850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:18:03.825268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:18:03.825268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:03.825268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:03.825268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:03.825268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:03.825268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:03.825268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:03.825268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:03.825268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:03.825268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:03.825268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:03.825268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:03.825268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:03.825268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:03.825268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:03.825268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:03.825268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:03.825268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:03.825268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:03.827297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:03.827297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:03.827297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.827297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:03.829998 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:03.830230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:03.830342 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:03.830473 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:03.830843 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:03.831108 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:03.831281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.831804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:03.831804 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:03.831804 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:03.831804 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:03.831804 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:03.831804 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:03.831804 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.831804 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:03.832683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.832683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:03.832683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:03.832683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:03.832683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.832683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:18:03.832683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:03.832683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:03.832683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:03.832683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:03.832683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:03.832683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:03.833240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:03.833240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:03.833240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:03.833240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:03.833240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:03.833240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:03.833240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:03.833240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:03.833240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:03.833240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:03.833240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:03.833240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:03.833240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:03.833480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.833480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:03.833769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.833769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:03.833769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:03.833769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:03.833769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:03.833769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:03.833769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.833769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:03.833769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:03.833769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.833769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:03.833769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:03.833769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.833769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:03.833769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:03.833769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:03.834060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:18:03.834060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:03.834060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:03.834060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:03.834060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:03.834060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:03.834060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:03.834060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:03.834060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:03.834060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:03.834060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:03.834060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:03.834060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:03.834060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:03.834060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:03.834060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:03.834060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:03.834060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:03.834060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:03.834373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:03.834373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:03.834373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:08.005728 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:08.006552 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:08.006844 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:08.008331 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:08.042244 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:08.042630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.043503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:08.043503 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:08.043503 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:08.043503 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:08.043503 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:08.043503 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:08.043503 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.043503 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:08.044424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.044424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:08.044424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:08.044424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:08.044424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.044424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:18:08.044424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:08.044424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:08.044424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:08.044424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:08.044424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:08.044424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:08.045088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:08.045088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:08.045088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:08.045088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:08.045088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:08.045088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:08.045088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:08.045088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:08.045088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:08.045088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:08.045088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:08.045088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:08.045088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:08.045669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.045669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:08.045669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.045669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:08.045669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:08.045669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:08.045669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:08.045669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:08.045669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.045669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:08.045669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:08.045669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.045669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:08.045669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:08.045669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.046395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:08.046395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:08.046395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.046395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:18:08.046395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:08.046395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:08.046395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:08.046395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:08.046395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:08.046395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:08.046395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:08.046395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:08.046395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:08.046395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:08.046395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:08.046395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:08.046395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:08.046395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:08.046395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:08.047237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:08.047237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:08.047237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:08.047237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:08.047237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.047237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:08.765841 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:08.767423 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:08.767423 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:08.768275 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:08.768708 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:08.768898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.769765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:08.769765 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:08.769765 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:08.769765 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:08.769765 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:08.769765 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:08.769765 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.769765 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:08.773703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.773703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:08.773703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:08.773703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:08.773703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.773703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:18:08.773703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:08.773703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:08.773703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:08.773703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:08.773703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:08.773703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:08.774194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:08.774194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:08.774194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:08.774194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:08.774194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:08.774194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:08.774194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:08.774194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:08.774194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:08.774194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:08.774194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:08.774194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:08.774194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:08.774575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.774575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:08.775064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.775064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:08.775064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:08.775064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:08.775064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:08.775064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:08.775064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.775064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:08.775064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:08.775064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.775064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:08.775064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:08.775064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.775064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:08.775064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:08.775064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.775663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:18:08.775663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:08.775663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:08.775663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:08.775663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:08.775663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:08.775663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:08.775663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:08.775663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:08.775663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:08.775663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:08.775663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:08.775663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:08.775663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:08.775663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:08.775663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:08.775663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:08.775663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:08.775663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:08.776203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:08.776203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.776203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:08.809783 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:08.810823 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:08.810823 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:08.810823 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:18:08.811518 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:08.811928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.813046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:08.813046 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:08.813046 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:08.813046 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:08.813046 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:08.813046 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:08.813046 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.813046 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:08.814796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.814796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:18:08.814796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:08.814796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:08.814796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:08.814796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:08.814796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:08.814796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:08.814796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:08.814796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:08.814796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:08.814796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:18:08.814796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:08.814796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:18:08.814796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:18:08.815307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:08.815307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:18:08.815307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:08.815307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:18:08.815307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:18:08.815307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:08.815307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:18:08.815307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.815307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:18:08.816193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.816193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:08.816193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:08.816193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:08.816193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.816193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:18:08.816193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:18:08.816193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:18:08.816193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:08.816193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:08.816193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:08.816193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:08.816584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:08.816584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:08.816584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:08.816584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:08.816584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:08.816584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:08.816584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:08.816584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:08.816584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:08.816584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:08.816584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:08.816584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:08.816584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:08.816584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:08.817007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:08.817007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:08.817007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.817007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:08.818022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.818022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:08.818022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:08.818022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:08.818022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:08.818022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:08.818022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.818022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:08.818022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:08.818022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.818022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:08.818022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:08.818022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.818022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:08.818022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:08.818022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.818502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:18:08.818502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:18:08.818502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:18:08.818502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:08.818502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:08.818502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:08.818502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:08.818502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:08.818502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:08.818502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:08.818502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:08.818502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:08.818502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:08.818502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:08.818502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:08.818502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:08.818502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:08.818502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:08.819038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:08.819038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:08.819038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:08.819038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:08.819038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:08.819038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.819038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:08.819669 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:08.819906 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:08.820023 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:08.820208 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:08.820786 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:08.821165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.822000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:08.822000 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:08.822000 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:08.822000 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:08.822000 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:08.822000 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:08.822000 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.822000 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:08.822812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.822812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:18:08.822812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:08.822812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:08.822812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:08.822812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:08.822812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:08.822812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:08.822812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:08.822812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:08.822812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:08.822812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:18:08.822812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:08.822812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:18:08.822812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:18:08.823222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:08.823222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:18:08.823222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:08.823222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:18:08.823222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:18:08.823222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:08.823222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:18:08.823222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.823222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:18:08.823662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.823662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:08.823662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:08.823662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:08.823662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.823662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:18:08.823662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:18:08.823662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:08.823662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:08.823662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:08.823662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:08.824010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:08.824010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:08.824010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:08.824010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:08.824010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:08.824010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:08.824010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:08.824010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:08.824010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:08.824010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:08.824010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:08.824010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:08.824010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:08.824010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:08.824387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:08.824387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:08.824387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.824387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:08.824824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.824824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:08.824824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:08.824824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:08.824824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:08.824824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:08.824824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.824824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:08.824824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:08.824824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.824824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:08.824824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:08.824824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.824824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:08.824824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:08.824824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.824824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:18:08.825316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:18:08.825316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:08.825316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:08.825316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:08.825316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:08.825316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:08.825316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:08.825316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:08.825316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:08.825316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:08.825316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:08.825316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:08.825316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:08.825316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:08.825316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:08.825316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:08.825316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:08.825316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:08.825316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:08.825850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:08.825850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:08.825850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.825850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:08.830300 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:08.830487 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:08.830594 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:08.830769 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:08.831265 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:08.831603 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:08.831901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.832710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:08.832710 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:08.832710 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:08.832710 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:08.832710 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:08.832710 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:08.832710 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.832710 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:08.833701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.833701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:08.833701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:08.833701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:08.833701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.833701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:18:08.833701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:08.833701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:08.833701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:08.833701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:08.833701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:08.833701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:08.834279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:08.834279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:08.834279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:08.834279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:08.834279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:08.834279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:08.834279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:08.834279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:08.834279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:08.834279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:08.834279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:08.834279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:08.834279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:08.834831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.834831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:08.834831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.834831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:08.834831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:08.834831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:08.834831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:08.834831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:08.834831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.834831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:08.834831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:08.834831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.834831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:08.834831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:08.834831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.835507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:08.835507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:08.835507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:08.835507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:18:08.835507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:08.835507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:08.835507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:08.835507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:08.835507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:08.835507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:08.835507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:08.835507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:08.835507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:08.835507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:08.835507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:08.835507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:08.835507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:08.835507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:08.835507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:08.836311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:08.836311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:08.836311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:08.836311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:08.836311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:08.836311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:13.006570 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:13.007434 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:13.007434 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:13.008469 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:13.044918 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:13.045317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.046252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:13.046252 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:13.046252 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:13.046252 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:13.046252 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:13.046252 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:13.046252 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.046252 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:13.047315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.047315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:13.047315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:13.047315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:13.047315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.047315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:18:13.047315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:13.047315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:13.047315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:13.047315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:13.047315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:13.047315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:13.047896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:13.047896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:13.047896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:13.047896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:13.047896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:13.047896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:13.047896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:13.047896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:13.047896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:13.047896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:13.047896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:13.047896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:13.047896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:13.048387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.048387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:13.048522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.048522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:13.048522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:13.048522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:13.048522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:13.048522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:13.048522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.048522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:13.048522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:13.048522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.048522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:13.048522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:13.048522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.048522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:13.048522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:13.048522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.049223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:18:13.049223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:13.049223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:13.049223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:13.049223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:13.049223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:13.049223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:13.049223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:13.049223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:13.049223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:13.049223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:13.049223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:13.049223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:13.049223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:13.049223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:13.049223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:13.049223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:13.049223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:13.049223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:13.050391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:13.050391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.050391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:13.767088 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:13.767570 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:13.767812 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:13.770565 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:13.770565 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:13.770565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.770882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:13.770882 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:13.770882 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:13.770882 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:13.770882 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:13.770882 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:13.770882 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.770882 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:13.772549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.772549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:13.772549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:13.772549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:13.772549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.772549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:18:13.772549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:13.772549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:13.772549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:13.772549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:13.772549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:13.772549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:13.773111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:13.773111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:13.773111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:13.773111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:13.773111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:13.773111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:13.773111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:13.773111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:13.773111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:13.773111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:13.773111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:13.773111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:13.773111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:13.773629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.773629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:13.774226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.774226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:13.774226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:13.774226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:13.774226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:13.774226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:13.774226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.774226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:13.774226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:13.774226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.774226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:13.774226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:13.774226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.774226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:13.774226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:13.774226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.774915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:18:13.774915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:13.774915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:13.774915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:13.774915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:13.774915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:13.774915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:13.774915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:13.774915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:13.774915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:13.774915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:13.774915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:13.774915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:13.774915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:13.774915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:13.774915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:13.774915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:13.774915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:13.774915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:13.775645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:13.775645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.775645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:13.811488 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:13.811911 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:13.812094 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:13.812297 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:18:13.813775 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:13.813775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.814050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:13.814050 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:13.814050 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:13.814050 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:13.814050 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:13.814050 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:13.814050 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.814050 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:13.815411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.815411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:18:13.815411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:13.815411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:13.815411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:13.815411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:13.815411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:13.815411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:13.815411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:13.815411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:13.815411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:13.815411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:18:13.815411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:13.815411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:18:13.815411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:18:13.816668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:13.816668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:18:13.816668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:13.816668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:18:13.816668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:18:13.816668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:13.816668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:18:13.816668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.816668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:18:13.817056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.817056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:13.817056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:13.817056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:13.817056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.817056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:18:13.817056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:18:13.817056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:18:13.817056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:13.817056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:13.817056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:13.817056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:13.817353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:13.817353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:13.817353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:13.817353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:13.817353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:13.817353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:13.817353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:13.817353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:13.817353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:13.817353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:13.817353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:13.817353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:13.817353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:13.817353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:13.817701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:13.817701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:13.817701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.817701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:13.819153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.819153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:13.819153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:13.819153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:13.819153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:13.819153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:13.819153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.819153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:13.819153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:13.819153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.819153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:13.819153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:13.819153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.819153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:13.819153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:13.819153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.819918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:18:13.819918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:18:13.819918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:18:13.819918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:13.819918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:13.819918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:13.819918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:13.819918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:13.819918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:13.819918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:13.819918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:13.819918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:13.819918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:13.819918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:13.819918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:13.819918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:13.819918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:13.819918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:13.820364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:13.820364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:13.820364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:13.820364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:13.820364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:13.820364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.820364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:13.820364 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:13.820364 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:13.820364 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:13.821130 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:13.821130 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:13.821256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.822012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:13.822012 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:13.822012 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:13.822012 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:13.822012 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:13.822012 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:13.822012 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.822012 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:13.822639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.822639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:18:13.822639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:13.822639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:13.822639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:13.822639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:13.822639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:13.822639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:13.822639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:13.822639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:13.822639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:13.822639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:18:13.822639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:13.822639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:18:13.822639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:18:13.823396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:13.823396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:18:13.823396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:13.823396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:18:13.823396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:18:13.823396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:13.823396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:18:13.823396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.823396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:18:13.823396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.823396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:13.823396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:13.823396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:13.823396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.823794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:18:13.823794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:18:13.823794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:13.823794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:13.823794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:13.823794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:13.823794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:13.823794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:13.823794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:13.823794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:13.823794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:13.823794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:13.823794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:13.824117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:13.824117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:13.824117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:13.824117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:13.824117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:13.824117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:13.824117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:13.824117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:13.824117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:13.824117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.824117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:13.824117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.824422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:13.824422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:13.824422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:13.824422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:13.824422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:13.824422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.824422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:13.824422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:13.824422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.824422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:13.824422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:13.824422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.824422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:13.824422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:13.824422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.824422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:18:13.824422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:18:13.824422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:13.824422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:13.824958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:13.824958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:13.824958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:13.824958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:13.824958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:13.824958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:13.824958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:13.824958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:13.824958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:13.824958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:13.824958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:13.824958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:13.824958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:13.824958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:13.824958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:13.824958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:13.824958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:13.824958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:13.824958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.824958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:13.830609 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:13.830848 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:13.830948 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:13.831136 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:13.831609 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:13.832005 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:13.832230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.833027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:13.833027 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:13.833027 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:13.833027 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:13.833027 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:13.833027 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:13.833027 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.833027 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:13.833898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.833898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:13.833898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:13.833898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:13.833898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.833898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:18:13.833898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:13.833898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:13.833898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:13.833898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:13.833898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:13.833898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:13.834409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:13.834409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:13.834409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:13.834409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:13.834409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:13.834409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:13.834409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:13.834409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:13.834409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:13.834409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:13.834409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:13.834409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:13.834409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:13.834946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.834946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:13.834946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.834946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:13.834946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:13.834946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:13.834946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:13.834946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:13.834946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.834946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:13.834946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:13.834946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.834946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:13.834946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:13.834946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.835535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:13.835535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:13.835535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:13.835535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:18:13.835535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:13.835535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:13.835535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:13.835535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:13.835535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:13.835535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:13.835535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:13.835535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:13.835535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:13.835535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:13.835535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:13.835535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:13.835535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:13.835535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:13.835535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:13.836306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:13.836306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:13.836306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:13.836306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:13.836306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:13.836306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:18.007353 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:18.008000 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:18.008000 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:18.009754 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:18.043239 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:18.043697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.044593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:18.044593 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:18.044593 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:18.044593 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:18.044593 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:18.044593 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:18.044593 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.044593 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:18.045505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.045505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:18.045505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:18.045505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:18.045505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.045505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:18:18.045505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:18.045505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:18.045505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:18.045505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:18.045505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:18.045505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:18.046085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:18.046085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:18.046085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:18.046085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:18.046085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:18.046085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:18.046085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:18.046085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:18.046085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:18.046085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:18.046085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:18.046085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:18.046085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:18.046677 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.046677 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:18.046677 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.046677 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:18.046677 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:18.046677 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:18.046677 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:18.046677 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:18.046677 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.046677 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:18.046677 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:18.046677 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.046677 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:18.046677 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:18.046677 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.047399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:18.047399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:18.047399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.047399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:18:18.047399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:18.047399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:18.047399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:18.047399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:18.047399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:18.047399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:18.047399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:18.047399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:18.047399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:18.047399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:18.047399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:18.047399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:18.047399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:18.047399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:18.047399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:18.048187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:18.048187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:18.048187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:18.048187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:18.048187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.048187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:18.768070 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:18.768645 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:18.768876 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:18.770846 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:18.771388 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:18.771751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.772978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:18.772978 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:18.772978 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:18.772978 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:18.772978 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:18.772978 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:18.772978 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.772978 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:18.774783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.774783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:18.774783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:18.774783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:18.774783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.774783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:18:18.774783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:18.774783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:18.774783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:18.774783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:18.774783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:18.774783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:18.775396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:18.775396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:18.775396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:18.775396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:18.775396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:18.775396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:18.775396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:18.775396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:18.775396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:18.775396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:18.775396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:18.775396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:18.775396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:18.775992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.775992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:18.776753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.776753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:18.776753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:18.776753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:18.776753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:18.776753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:18.776753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.776753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:18.776753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:18.776753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.776753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:18.776753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:18.776753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.776753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:18.776753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:18.776753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.777465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:18:18.777465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:18.777465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:18.777465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:18.777465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:18.777465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:18.777465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:18.777465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:18.777465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:18.777465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:18.777465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:18.777465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:18.777465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:18.777465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:18.777465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:18.777465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:18.777465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:18.777465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:18.777465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:18.778362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:18.778362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.778362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:18.812412 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:18.812886 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:18.813077 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:18.813280 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:18:18.814096 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:18.814550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.817001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:18.817001 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:18.817001 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:18.817001 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:18.817001 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:18.817001 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:18.817001 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.817001 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:18.819124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.819124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:18:18.819124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:18.819124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:18.819124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:18.819124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:18.819124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:18.819124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:18.819124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:18.819124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:18.819124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:18.819124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:18:18.819124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:18.819124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:18:18.819124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:18:18.820240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:18.820240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:18:18.820240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:18.820240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:18:18.820240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:18:18.820240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:18.820240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:18:18.820240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.820240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:18:18.820600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.820600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:18.820600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:18.820600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:18.820600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.820600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:18:18.820600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:18:18.820600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:18:18.820600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:18.820600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:18.820600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:18.820600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:18.821178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:18.821178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:18.821178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:18.821178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:18.821178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:18.821178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:18.821178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:18.821178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:18.821178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:18.821178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:18.821178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:18.821178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:18.821178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:18.821178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:18.822159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:18.822159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:18.822159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.822159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:18.822159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.822159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:18.822159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:18.822159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:18.822159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:18.822159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:18.822159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.822159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:18.822159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:18.822159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.823659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:18.823659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:18.823659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.823659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:18.823659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:18.823659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.823659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:18:18.823659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:18:18.823659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:18:18.823659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:18.823659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:18.823659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:18.823659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:18.823659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:18.823659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:18.823659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:18.823659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:18.823659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:18.823659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:18.825222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:18.825222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:18.825222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:18.825222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:18.825222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:18.825222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:18.825222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:18.825222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:18.825222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:18.825222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:18.825222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.825222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:18.825222 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:18.825222 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:18.826647 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:18.826647 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:18.826647 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:18.826647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.826647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:18.826647 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:18.826647 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:18.826647 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:18.826647 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:18.826647 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:18.826647 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.826647 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:18.827145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.827145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:18:18.827145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:18.827145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:18.827145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:18.827145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:18.827145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:18.827145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:18.827145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:18.827145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:18.827145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:18.827145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:18:18.827145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:18.827145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:18:18.827145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:18:18.827828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:18.827828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:18:18.827828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:18.827828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:18:18.827828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:18:18.827828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:18.827828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:18:18.827828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.827828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:18:18.827828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.827828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:18.827828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:18.827828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:18.827828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.828319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:18:18.828319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:18:18.828319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:18.828319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:18.828319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:18.828319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:18.828319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:18.828319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:18.828319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:18.828319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:18.828319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:18.828319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:18.828319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:18.828745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:18.828745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:18.828745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:18.828745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:18.828745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:18.828745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:18.828745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:18.828745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:18.828745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:18.828745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.828745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:18.828745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.829215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:18.829215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:18.829215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:18.829215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:18.829215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:18.829215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.829215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:18.829215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:18.829215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.829215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:18.829215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:18.829215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.829215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:18.829215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:18.829215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.829215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:18:18.829215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:18:18.829215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:18.829215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:18.829853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:18.829853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:18.829853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:18.829853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:18.829853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:18.829853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:18.829853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:18.829853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:18.829853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:18.829853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:18.829853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:18.829853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:18.829853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:18.829853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:18.829853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:18.829853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:18.829853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:18.829853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:18.829853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.829853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:18.831061 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:18.831484 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:18.831662 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:18.831945 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:18.832390 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:18.832685 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:18.832857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.833524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:18.833524 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:18.833524 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:18.833524 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:18.833524 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:18.833524 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:18.833524 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.833524 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:18.835023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.835023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:18.835023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:18.835023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:18.835023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.835023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:18:18.835023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:18.835023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:18.835023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:18.835023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:18.835023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:18.835023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:18.835437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:18.835437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:18.835437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:18.835437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:18.835437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:18.835437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:18.835437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:18.835437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:18.835437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:18.835437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:18.835437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:18.835437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:18.835437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:18.835771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.835771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:18.836520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.836520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:18.836520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:18.836520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:18.836520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:18.836520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:18.836520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.836520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:18.836520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:18.836520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.836520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:18.836520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:18.836520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.836520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:18.836520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:18.836520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:18.837245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:18:18.837245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:18.837245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:18.837245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:18.837245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:18.837245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:18.837245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:18.837245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:18.837245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:18.837245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:18.837245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:18.837245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:18.837245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:18.837245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:18.837245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:18.837245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:18.837245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:18.837245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:18.837245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:18.837962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:18.837962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:18.837962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:23.008288 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:23.008946 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:23.008946 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:23.010326 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:23.048610 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:23.049161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.050375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:23.050375 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:23.050375 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:23.050375 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:23.050375 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:23.050375 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:23.050375 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.050375 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:23.051365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.051365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:23.051365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:23.051365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:23.051365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.051365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:18:23.051365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:23.051365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:23.051365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:23.051365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:23.051365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:23.051365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:23.051885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:23.051885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:23.051885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:23.051885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:23.051885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:23.051885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:23.051885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:23.051885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:23.051885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:23.051885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:23.051885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:23.051885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:23.051885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:23.052310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.052310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:23.052310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.052310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:23.052310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:23.052310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:23.052310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:23.052310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:23.052310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.052310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:23.052310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:23.052310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.052310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:23.052310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:23.052310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.052804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:23.052804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:23.052804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.052804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:18:23.052804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:23.052804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:23.052804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:23.052804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:23.052804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:23.052804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:23.052804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:23.052804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:23.052804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:23.052804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:23.052804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:23.052804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:23.052804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:23.052804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:23.052804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:23.053763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:23.053763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:23.053763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:23.053763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:23.053763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.053763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:23.769070 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:23.769600 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:23.769818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:23.771383 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:23.771896 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:23.772228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.773362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:23.773362 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:23.773362 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:23.773362 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:23.773362 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:23.773362 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:23.773362 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.773362 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:23.775296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.775296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:23.775296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:23.775296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:23.775296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.775296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:18:23.775296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:23.775296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:23.775296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:23.775296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:23.775296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:23.775296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:23.775873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:23.775873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:23.775873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:23.775873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:23.775873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:23.775873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:23.775873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:23.775873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:23.775873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:23.775873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:23.775873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:23.775873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:23.775873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:23.776390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.776390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:23.777170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.777170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:23.777170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:23.777170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:23.777170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:23.777170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:23.777170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.777170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:23.777170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:23.777170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.777170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:23.777170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:23.777170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.777170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:23.777170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:23.777170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.777659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:18:23.777659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:23.777659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:23.777659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:23.777659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:23.777659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:23.777659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:23.777659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:23.777659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:23.777659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:23.777659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:23.777659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:23.777659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:23.777659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:23.777659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:23.777659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:23.777659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:23.777659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:23.777659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:23.778134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:23.778134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.778134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:23.814140 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:23.814140 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:23.814140 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:23.814140 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:18:23.814926 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:23.815306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.816217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:23.816217 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:23.816217 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:23.816217 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:23.816217 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:23.816217 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:23.816217 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.816217 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:23.818091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.818091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:18:23.818091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:23.818091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:23.818091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:23.818091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:23.818091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:23.818091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:23.818091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:23.818091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:23.818091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:23.818091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:18:23.818091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:23.818091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:18:23.818091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:18:23.819493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:23.819493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:18:23.819493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:23.819493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:18:23.819493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:18:23.819493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:23.819493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:18:23.819493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.819493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:18:23.819493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.819493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:23.819493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:23.819493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:23.819493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.820705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:18:23.820705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:18:23.820705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:18:23.820705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:23.820705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:23.820705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:23.820705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:23.820705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:23.820705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:23.820705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:23.820705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:23.820705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:23.820705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:23.821263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:23.821263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:23.821263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:23.821263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:23.821263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:23.821263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:23.821263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:23.821263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:23.821263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:23.821263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:23.821263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.821263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:23.822090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.822090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:23.822090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:23.822090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:23.822090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:23.822090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:23.822090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.822090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:23.822090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:23.822090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.822090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:23.822090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:23.822090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.822090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:23.822090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:23.822090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.822867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:18:23.822867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:18:23.822867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:18:23.822867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:23.822867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:23.822867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:23.822867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:23.822867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:23.822867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:23.822867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:23.822867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:23.822867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:23.822867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:23.822867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:23.822867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:23.822867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:23.822867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:23.822867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:23.823576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:23.823576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:23.823576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:23.823576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:23.823576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:23.823576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.823576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:23.823576 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:23.823576 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:23.823576 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:23.823822 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:23.823822 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:23.823822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.824092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:23.824092 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:23.824092 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:23.824092 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:23.824092 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:23.824092 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:23.824092 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.824092 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:23.824866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.824866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:18:23.824866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:23.824866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:23.824866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:23.824866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:23.824866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:23.824866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:23.824866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:23.824866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:23.824866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:23.824866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:18:23.824866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:23.824866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:18:23.824866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:18:23.825152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:23.825152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:18:23.825152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:23.825152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:18:23.825152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:18:23.825152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:23.825152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:18:23.825152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.825152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:18:23.825713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.825713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:23.825713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:23.825713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:23.825713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.825713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:18:23.825713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:18:23.825713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:23.825713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:23.825713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:23.825713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:23.825965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:23.825965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:23.825965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:23.825965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:23.825965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:23.825965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:23.825965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:23.825965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:23.825965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:23.825965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:23.825965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:23.825965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:23.825965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:23.825965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:23.826276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:23.826276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:23.826276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.826276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:23.826887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.826887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:23.826887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:23.826887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:23.826887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:23.826887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:23.826887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.826887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:23.826887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:23.826887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.826887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:23.826887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:23.826887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.826887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:23.826887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:23.826887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.826887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:18:23.827229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:18:23.827229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:23.827229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:23.827229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:23.827229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:23.827229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:23.827229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:23.827229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:23.827229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:23.827229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:23.827229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:23.827229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:23.827229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:23.827229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:23.827229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:23.827229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:23.827229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:23.827229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:23.827229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:23.827568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:23.827568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:23.827568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.827568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:23.831416 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:23.831616 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:23.831746 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:23.831897 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:23.832384 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:23.832772 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:23.833012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.833770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:23.833770 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:23.833770 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:23.833770 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:23.833770 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:23.833770 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:23.833770 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.833770 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:23.834692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.834692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:23.834692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:23.834692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:23.834692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.834692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:18:23.834692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:23.834692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:23.834692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:23.834692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:23.834692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:23.834692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:23.835017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:23.835017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:23.835017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:23.835017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:23.835017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:23.835017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:23.835017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:23.835017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:23.835017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:23.835017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:23.835017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:23.835017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:23.835017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:23.835341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.835341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:23.835632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.835632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:23.835632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:23.835632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:23.835632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:23.835632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:23.835632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.835632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:23.835632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:23.835632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.835632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:23.835632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:23.835632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.835632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:23.835632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:23.835632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:23.836055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:18:23.836055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:23.836055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:23.836055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:23.836055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:23.836055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:23.836055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:23.836055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:23.836055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:23.836055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:23.836055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:23.836055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:23.836055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:23.836055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:23.836055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:23.836055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:23.836055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:23.836055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:23.836055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:23.836507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:23.836507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:23.836507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:28.008843 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:28.009247 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:28.009247 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:28.010153 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:28.048234 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:28.048626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.049543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:28.049543 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:28.049543 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:28.049543 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:28.049543 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:28.049543 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:28.049543 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.049543 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:28.050738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.050738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:28.050738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:28.050738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:28.050738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.050738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:18:28.050738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:28.050738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:28.050738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:28.050738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:28.050738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:28.050738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:28.051076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:28.051076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:28.051076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:28.051076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:28.051076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:28.051076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:28.051076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:28.051076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:28.051076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:28.051076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:28.051076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:28.051076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:28.051076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:28.051494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.051494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:28.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:28.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:28.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:28.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:28.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:28.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:28.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:28.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:28.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:28.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:28.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:28.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.052343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:18:28.052343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:28.052343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:28.052343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:28.052343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:28.052343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:28.052343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:28.052343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:28.052343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:28.052343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:28.052343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:28.052343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:28.052343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:28.052343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:28.052343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:28.052343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:28.052343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:28.052343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:28.052343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:28.052877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:28.052877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.052877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:28.769783 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:28.770209 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:28.770379 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:28.771575 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:28.772071 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:28.772375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.773361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:28.773361 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:28.773361 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:28.773361 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:28.773361 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:28.773361 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:28.773361 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.773361 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:28.774975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.774975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:28.774975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:28.774975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:28.774975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.774975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:18:28.774975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:28.774975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:28.774975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:28.774975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:28.774975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:28.774975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:28.775532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:28.775532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:28.775532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:28.775532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:28.775532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:28.775532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:28.775532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:28.775532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:28.775532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:28.775532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:28.775532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:28.775532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:28.775532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:28.776206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.776206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:28.776845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.776845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:28.776845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:28.776845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:28.776845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:28.776845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:28.776845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.776845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:28.776845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:28.776845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.776845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:28.776845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:28.776845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.776845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:28.776845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:28.776845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.777535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:18:28.777535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:28.777535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:28.777535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:28.777535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:28.777535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:28.777535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:28.777535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:28.777535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:28.777535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:28.777535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:28.777535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:28.777535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:28.777535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:28.777535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:28.777535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:28.777535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:28.777535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:28.777535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:28.778387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:28.778387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.778387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:28.813632 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:28.814080 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:28.814318 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:28.814567 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:18:28.815175 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:28.815556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.816477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:28.816477 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:28.816477 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:28.816477 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:28.816477 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:28.816477 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:28.816477 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.816477 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:28.817794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.817794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:18:28.817794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:28.817794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:28.817794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:28.817794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:28.817794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:28.817794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:28.817794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:28.817794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:28.817794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:28.817794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:18:28.817794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:28.817794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:18:28.817794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:18:28.818487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:28.818487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:18:28.818487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:28.818487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:18:28.818487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:18:28.818487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:28.818487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:18:28.818487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.818487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:18:28.819080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.819080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:28.819080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:28.819080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:28.819080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.819080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:18:28.819080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:18:28.819080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:18:28.819080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:28.819080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:28.819080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:28.819080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:28.819599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:28.819599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:28.819599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:28.819599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:28.819599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:28.819599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:28.819599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:28.819599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:28.819599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:28.819599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:28.819599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:28.819599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:28.819599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:28.819599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:28.820424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:28.820424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:28.820424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.820424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:28.820713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.820713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:28.820713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:28.820713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:28.820713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:28.820713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:28.820713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.820713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:28.820713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:28.820713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.820713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:28.820713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:28.820713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.820713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:28.820713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:28.820713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.821478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:18:28.821478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:18:28.821478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:18:28.821478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:28.821478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:28.821478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:28.821478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:28.821478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:28.821478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:28.821478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:28.821478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:28.821478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:28.821478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:28.821478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:28.821478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:28.821478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:28.821478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:28.821478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:28.822327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:28.822327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:28.822327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:28.822327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:28.822327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:28.822327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.822327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:28.822327 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:28.822327 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:28.822815 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:28.822815 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:28.823151 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:28.823502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.824490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:28.824490 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:28.824490 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:28.824490 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:28.824490 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:28.824490 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:28.824490 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.824490 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:28.825363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.825363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:18:28.825363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:28.825363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:28.825363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:28.825363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:28.825363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:28.825363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:28.825363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:28.825363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:28.825363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:28.825363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:18:28.825363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:28.825363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:18:28.825363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:18:28.825837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:28.825837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:18:28.825837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:28.825837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:18:28.825837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:18:28.825837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:28.825837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:18:28.825837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.825837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:18:28.826303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.826303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:28.826303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:28.826303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:28.826303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.826303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:18:28.826303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:18:28.826303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:28.826303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:28.826303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:28.826303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:28.826723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:28.826723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:28.826723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:28.826723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:28.826723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:28.826723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:28.826723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:28.826723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:28.826723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:28.826723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:28.826723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:28.826723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:28.826723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:28.826723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:28.827240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:28.827240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:28.827240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.827240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:28.827394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.827394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:28.827394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:28.827394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:28.827394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:28.827394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:28.827394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.827394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:28.827394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:28.827394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.827394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:28.827394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:28.827394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.827394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:28.827394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:28.827394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.827394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:18:28.828081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:18:28.828081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:28.828081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:28.828081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:28.828081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:28.828081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:28.828081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:28.828081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:28.828081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:28.828081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:28.828081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:28.828081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:28.828081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:28.828081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:28.828081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:28.828081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:28.828081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:28.828081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:28.828081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:28.828769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:28.828769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:28.828769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.828769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:28.831707 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:28.832097 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:28.832190 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:28.832354 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:28.832893 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:28.833391 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:28.833725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.836088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:28.836088 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:28.836088 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:28.836088 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:28.836088 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:28.836088 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:28.836088 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.836088 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:28.837131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.837131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:28.837131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:28.837131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:28.837131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.837131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:18:28.837131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:28.837131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:28.837131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:28.837131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:28.837131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:28.837131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:28.837465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:28.837465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:28.837465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:28.837465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:28.837465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:28.837465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:28.837465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:28.837465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:28.837465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:28.837465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:28.837465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:28.837465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:28.837465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:28.838111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.838111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:28.838816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.838816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:28.838816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:28.838816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:28.838816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:28.838816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:28.838816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.838816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:28.838816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:28.838816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.838816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:28.838816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:28.838816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.838816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:28.838816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:28.838816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:28.839512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:18:28.839512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:28.839512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:28.839512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:28.839512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:28.839512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:28.839512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:28.839512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:28.839512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:28.839512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:28.839512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:28.839512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:28.839512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:28.839512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:28.839512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:28.839512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:28.839512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:28.839512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:28.839512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:28.840045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:28.840045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:28.840045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:33.012402 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:33.013445 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:33.013445 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:33.014936 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:33.051284 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:33.051576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.052351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:33.052351 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:33.052351 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:33.052351 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:33.052351 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:33.052351 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:33.052351 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.052351 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:33.053228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.053228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:33.053228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:33.053228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:33.053228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.053228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:18:33.053228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:33.053228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:33.053228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:33.053228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:33.053228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:33.053228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:33.053615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:33.053615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:33.053615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:33.053615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:33.053615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:33.053615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:33.053615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:33.053615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:33.053615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:33.053615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:33.053615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:33.053615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:33.053615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:33.054008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.054008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:33.054281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.054281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:33.054281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:33.054281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:33.054281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:33.054281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:33.054281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.054281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:33.054281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:33.054281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.054281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:33.054281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:33.054281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.054281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:33.054281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:33.054281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.054618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:18:33.054618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:33.054618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:33.054618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:33.054618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:33.054618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:33.054618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:33.054618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:33.054618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:33.054618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:33.054618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:33.054618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:33.054618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:33.054618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:33.054618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:33.054618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:33.054618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:33.054618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:33.054618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:33.055015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:33.055015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.055015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:33.770490 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:33.771191 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:33.771396 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:33.772627 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:33.773067 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:33.773307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.774248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:33.774248 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:33.774248 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:33.774248 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:33.774248 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:33.774248 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:33.774248 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.774248 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:33.775945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.775945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:33.775945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:33.775945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:33.775945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.775945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:18:33.775945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:33.775945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:33.775945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:33.775945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:33.775945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:33.775945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:33.776520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:33.776520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:33.776520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:33.776520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:33.776520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:33.776520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:33.776520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:33.776520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:33.776520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:33.776520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:33.776520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:33.776520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:33.776520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:33.777126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.777126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:33.777705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.777705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:33.777705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:33.777705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:33.777705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:33.777705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:33.777705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.777705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:33.777705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:33.777705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.777705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:33.777705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:33.777705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.777705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:33.777705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:33.777705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.778400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:18:33.778400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:33.778400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:33.778400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:33.778400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:33.778400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:33.778400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:33.778400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:33.778400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:33.778400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:33.778400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:33.778400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:33.778400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:33.778400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:33.778400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:33.778400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:33.778400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:33.778400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:33.778400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:33.779231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:33.779231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.779231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:33.814216 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:33.814866 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:33.814866 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:33.815009 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:18:33.815367 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:33.815894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.816361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:33.816361 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:33.816361 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:33.816361 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:33.816361 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:33.816361 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:33.816361 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.816361 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:33.817508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.817508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:18:33.817508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:33.817508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:33.817508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:33.817508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:33.817508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:33.817508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:33.817508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:33.817508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:33.817508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:33.817508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:18:33.817508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:33.817508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:18:33.817508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:18:33.818004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:33.818004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:18:33.818004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:33.818004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:18:33.818004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:18:33.818004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:33.818004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:18:33.818004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.818004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:18:33.818869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.818869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:33.818869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:33.818869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:33.818869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.818869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:18:33.818869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:18:33.818869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:18:33.818869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:33.818869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:33.818869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:33.818869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:33.819254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:33.819254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:33.819254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:33.819254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:33.819254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:33.819254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:33.819254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:33.819254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:33.819254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:33.819254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:33.819254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:33.819254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:33.819254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:33.819254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:33.819643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:33.819643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:33.819643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.819643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:33.820584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.820584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:33.820584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:33.820584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:33.820584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:33.820584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:33.820584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.820584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:33.820584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:33.820584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.820584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:33.820584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:33.820584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.820584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:33.820584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:33.820584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.821712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:18:33.821712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:18:33.821712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:18:33.821712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:33.821712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:33.821712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:33.821712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:33.821712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:33.821712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:33.821712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:33.821712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:33.821712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:33.821712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:33.821712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:33.821712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:33.821712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:33.821712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:33.821712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:33.822230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:33.822230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:33.822230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:33.822230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:33.822230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:33.822230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.822230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:33.822565 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:33.823416 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:33.823416 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:33.823416 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:33.823557 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:33.823956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.824459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:33.824459 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:33.824459 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:33.824459 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:33.824459 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:33.824459 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:33.824459 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.824459 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:33.825353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.825353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:18:33.825353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:33.825353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:33.825353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:33.825353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:33.825353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:33.825353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:33.825353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:33.825353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:33.825353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:33.825353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:18:33.825353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:33.825353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:18:33.825353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:18:33.826627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:33.826627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:18:33.826627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:33.826627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:18:33.826627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:18:33.826627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:33.826627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:18:33.826627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.826627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:18:33.826627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.826627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:33.826627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:33.826627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:33.826627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.827089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:18:33.827089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:18:33.827089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:33.827089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:33.827089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:33.827089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:33.827089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:33.827089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:33.827089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:33.827089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:33.827089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:33.827089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:33.827089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:33.827472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:33.827472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:33.827472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:33.827472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:33.827472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:33.827472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:33.827472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:33.827472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:33.827472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:33.827472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.827472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:33.827472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.827907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:33.827907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:33.827907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:33.827907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:33.827907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:33.827907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.827907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:33.827907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:33.827907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.827907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:33.827907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:33.827907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.827907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:33.827907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:33.827907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.827907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:18:33.827907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:18:33.827907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:33.827907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:33.828437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:33.828437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:33.828437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:33.828437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:33.828437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:33.828437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:33.828437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:33.828437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:33.828437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:33.828437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:33.828437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:33.828437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:33.828437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:33.828437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:33.828437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:33.828437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:33.828437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:33.828437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:33.828437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.828437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:33.832200 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:33.832477 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:33.832590 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:33.832765 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:33.833220 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:33.833608 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:33.833877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.834590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:33.834590 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:33.834590 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:33.834590 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:33.834590 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:33.834590 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:33.834590 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.834590 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:33.835421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.835421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:33.835421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:33.835421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:33.835421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.835421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:18:33.835421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:33.835421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:33.835421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:33.835421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:33.835421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:33.835421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:33.835819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:33.835819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:33.835819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:33.835819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:33.835819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:33.835819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:33.835819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:33.835819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:33.835819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:33.835819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:33.835819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:33.835819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:33.835819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:33.836209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.836209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:33.836389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.836389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:33.836389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:33.836389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:33.836389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:33.836389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:33.836389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.836389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:33.836389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:33.836389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.836389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:33.836389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:33.836389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.836389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:33.836389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:33.836389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:33.836891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:18:33.836891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:33.836891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:33.836891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:33.836891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:33.836891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:33.836891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:33.836891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:33.836891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:33.836891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:33.836891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:33.836891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:33.836891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:33.836891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:33.836891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:33.836891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:33.836891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:33.836891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:33.836891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:33.837436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:33.837436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:33.837436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:38.012976 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:38.014394 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:38.014394 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:38.014394 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:38.050001 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:38.050372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.051244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:38.051244 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:38.051244 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:38.051244 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:38.051244 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:38.051244 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:38.051244 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.051244 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:38.052074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.052074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:38.052074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:38.052074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:38.052074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.052074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:18:38.052074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:38.052074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:38.052074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:38.052074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:38.052074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:38.052074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:38.052766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:38.052766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:38.052766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:38.052766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:38.052766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:38.052766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:38.052766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:38.052766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:38.052766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:38.052766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:38.052766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:38.052766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:38.052766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:38.053321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.053321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:38.053321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.053321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:38.053321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:38.053321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:38.053321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:38.053321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:38.053321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.053321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:38.053321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:38.053321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.053321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:38.053321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:38.053321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.054027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:38.054027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:38.054027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.054027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:18:38.054027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:38.054027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:38.054027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:38.054027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:38.054027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:38.054027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:38.054027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:38.054027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:38.054027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:38.054027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:38.054027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:38.054027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:38.054027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:38.054027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:38.054027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:38.054967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:38.054967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:38.054967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:38.054967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:38.054967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.054967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:38.771237 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:38.771582 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:38.771668 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:38.772863 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:38.773347 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:38.773625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.774702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:38.774702 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:38.774702 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:38.774702 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:38.774702 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:38.774702 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:38.774702 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.774702 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:38.776370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.776370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:38.776370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:38.776370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:38.776370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.776370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:18:38.776370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:38.776370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:38.776370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:38.776370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:38.776370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:38.776370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:38.777135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:38.777135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:38.777135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:38.777135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:38.777135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:38.777135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:38.777135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:38.777135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:38.777135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:38.777135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:38.777135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:38.777135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:38.777135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:38.777950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.777950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:38.778108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.778108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:38.778108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:38.778108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:38.778108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:38.778108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:38.778108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.778108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:38.778108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:38.778108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.778108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:38.778108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:38.778108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.778108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:38.778108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:38.778108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.779437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:18:38.779437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:38.779437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:38.779437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:38.779437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:38.779437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:38.779437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:38.779437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:38.779437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:38.779437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:38.779437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:38.779437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:38.779437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:38.779437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:38.779437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:38.779437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:38.779437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:38.779437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:38.779437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:38.782412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:38.782412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.782412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:38.814530 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:38.814999 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:38.815134 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:38.815296 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:18:38.819074 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:38.819490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.820553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:38.820553 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:38.820553 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:38.820553 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:38.820553 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:38.820553 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:38.820553 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.820553 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:38.821834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.821834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:18:38.821834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:38.821834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:38.821834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:38.821834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:38.821834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:38.821834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:38.821834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:38.821834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:38.821834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:38.821834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:18:38.821834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:38.821834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:18:38.821834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:18:38.822828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:38.822828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:18:38.822828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:38.822828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:18:38.822828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:18:38.822828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:38.822828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:18:38.822828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.822828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:18:38.823249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.007 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.823249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:38.823249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:38.823249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:38.823249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.823249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:18:38.823249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:18:38.823249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:18:38.823249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:38.823249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:38.823249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:38.823249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:38.823840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:38.823840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:38.823840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:38.823840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:38.823840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:38.823840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:38.823840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:38.823840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:38.823840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:38.823840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:38.823840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:38.823840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:38.823840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:38.823840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:38.824457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:38.824457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:38.824457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.824457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:38.824643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.824643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:38.824643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:38.824643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:38.824643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:38.824643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:38.824643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.824643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:38.824643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:38.824643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.824643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:38.824643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:38.824643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.824643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:38.824643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:38.824643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.826694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:18:38.826694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:18:38.826694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:18:38.826694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:38.826694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:38.826694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:38.826694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:38.826694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:38.826694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:38.826694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:38.826694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:38.826694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:38.826694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:38.826694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:38.826694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:38.826694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:38.826694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:38.826694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:38.828195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:38.828195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:38.828195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:38.828195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:38.828195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:38.828195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.828195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:38.828195 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:38.828195 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:38.828195 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:38.828481 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:38.828481 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:38.828481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.828481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:38.828481 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:38.828481 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:38.828481 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:38.828481 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:38.828481 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:38.828481 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.828481 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:38.828848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.828848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:18:38.828848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:38.828848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:38.828848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:38.828848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:38.828848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:38.828848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:38.828848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:38.828848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:38.828848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:38.828848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:18:38.828848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:38.828848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:18:38.828848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:18:38.829212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:38.829212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:18:38.829212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:38.829212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:18:38.829212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:18:38.829212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:38.829212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:18:38.829212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.829212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:18:38.829996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.829996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:38.829996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:38.829996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:38.829996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.829996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:18:38.829996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:18:38.829996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:38.829996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:38.829996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:38.829996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:38.830357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:38.830357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:38.830357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:38.830357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:38.830357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:38.830357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:38.830357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:38.830357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:38.830357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:38.830357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:38.830357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:38.830357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:38.830357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:38.830357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:38.830850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:38.830850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:38.830850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.830850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:38.831039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.831039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:38.831039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:38.831039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:38.831039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:38.831039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:38.831039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.831039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:38.831039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:38.831039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.831039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:38.831039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:38.831039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.831039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:38.831039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:38.831039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.831039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:18:38.831894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:18:38.831894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:38.831894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:38.831894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:38.831894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:38.831894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:38.831894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:38.831894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:38.831894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:38.831894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:38.831894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:38.831894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:38.831894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:38.831894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:38.831894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:38.831894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:38.831894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:38.831894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:38.831894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:38.832908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:38.832908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:38.832908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.832908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:38.832908 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:38.832908 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:38.832908 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:38.832908 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:38.833550 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:38.833923 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:38.834103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.834809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:38.834809 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:38.834809 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:38.834809 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:38.834809 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:38.834809 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:38.834809 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.834809 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:38.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:38.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:38.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:38.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:18:38.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:38.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:38.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:38.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:38.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:38.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:38.835777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:38.835777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:38.835777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:38.835777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:38.835777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:38.835777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:38.835777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:38.835777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:38.835777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:38.835777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:38.835777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:38.835777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:38.835777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:38.836069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.836069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:38.836219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.836219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:38.836219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:38.836219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:38.836219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:38.836219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:38.836219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.836219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:38.836219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:38.836219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.836219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:38.836219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:38.836219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.836219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:38.836219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:38.836219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:38.836558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:18:38.836558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:38.836558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:38.836558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:38.836558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:38.836558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:38.836558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:38.836558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:38.836558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:38.836558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:38.836558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:38.836558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:38.836558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:38.836558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:38.836558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:38.836558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:38.836558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:38.836558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:38.836558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:38.836948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:38.836948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:38.836948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:43.013442 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:43.014248 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:43.014248 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:43.014991 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:43.052935 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:43.053434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.054391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:43.054391 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:43.054391 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:43.054391 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:43.054391 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:43.054391 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:43.054391 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.054391 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:43.055249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.055249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:43.055249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:43.055249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:43.055249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.055249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:18:43.055249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:43.055249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:43.055249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:43.055249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:43.055249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:43.055249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:43.055880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:43.055880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:43.055880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:43.055880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:43.055880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:43.055880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:43.055880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:43.055880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:43.055880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:43.055880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:43.055880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:43.055880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:43.055880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:43.056540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.056540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:43.056540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.056540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:43.056540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:43.056540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:43.056540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:43.056540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:43.056540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.056540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:43.056540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:43.056540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.056540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:43.056540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:43.056540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.057292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:43.057292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:43.057292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.057292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:18:43.057292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:43.057292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:43.057292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:43.057292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:43.057292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:43.057292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:43.057292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:43.057292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:43.057292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:43.057292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:43.057292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:43.057292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:43.057292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:43.057292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:43.057292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:43.058184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:43.058184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:43.058184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:43.058184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:43.058184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.058184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:43.771996 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:43.772893 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:43.773131 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:43.774429 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:43.774814 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:43.775052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.775971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:43.775971 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:43.775971 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:43.775971 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:43.775971 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:43.775971 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:43.775971 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.775971 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:43.779145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.779145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:43.779145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:43.779145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:43.779145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.779145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:18:43.779145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:43.779145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:43.779145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:43.779145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:43.779145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:43.779145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:43.779839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:43.779839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:43.779839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:43.779839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:43.779839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:43.779839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:43.779839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:43.779839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:43.779839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:43.779839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:43.779839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:43.779839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:43.779839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:43.780418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.780418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:43.780418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.780418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:43.780418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:43.780418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:43.780418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:43.780418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:43.780418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.780418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:43.780418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:43.780418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.780418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:43.780418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:43.780418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.781360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:43.781360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:43.781360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.781360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:18:43.781360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:43.781360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:43.781360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:43.781360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:43.781360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:43.781360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:43.781360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:43.781360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:43.781360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:43.781360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:43.781360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:43.781360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:43.781360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:43.781360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:43.781360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:43.783018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:43.783018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:43.783018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:43.783018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:43.783018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.783018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:43.815299 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:43.815874 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:43.816068 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:43.816267 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:18:43.817007 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:43.817331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.818182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:43.818182 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:43.818182 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:43.818182 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:43.818182 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:43.818182 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:43.818182 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.818182 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:43.819801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.819801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:18:43.819801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:43.819801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:43.819801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:43.819801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:43.819801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:43.819801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:43.819801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:43.819801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:43.819801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:43.819801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:18:43.819801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:43.819801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:18:43.819801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:18:43.820569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:43.820569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:18:43.820569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:43.820569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:18:43.820569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:18:43.820569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:43.820569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:18:43.820569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.820569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:18:43.821373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.821373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:43.821373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:43.821373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:43.821373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.821373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:18:43.821373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:18:43.821373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:18:43.821373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:43.821373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:43.821373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:43.821373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:43.822044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:43.822044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:43.822044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:43.822044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:43.822044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:43.822044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:43.822044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:43.822044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:43.822044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:43.822044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:43.822044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:43.822044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:43.822044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:43.822044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:43.822924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:43.822924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:43.822924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.822924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:43.822924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.822924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:43.822924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:43.822924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:43.822924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:43.822924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:43.822924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.822924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:43.822924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:43.822924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.823644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:43.823644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:43.823644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.823644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:43.823644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:43.823644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.823644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:18:43.823644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:18:43.823644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:18:43.823644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:43.823644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:43.823644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:43.823644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:43.823644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:43.823644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:43.823644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:43.823644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:43.823644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:43.823644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:43.824614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:43.824614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:43.824614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:43.824614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:43.824614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:43.824614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:43.824614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:43.824614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:43.824614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:43.824614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:43.824614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.824614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:43.825262 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:43.825638 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:43.825797 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:43.825936 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:43.826510 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:43.827072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.828051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:43.828051 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:43.828051 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:43.828051 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:43.828051 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:43.828051 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:43.828051 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.828051 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:43.829022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.829022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:18:43.829022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:43.829022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:43.829022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:43.829022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:43.829022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:43.829022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:43.829022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:43.829022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:43.829022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:43.829022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:18:43.829022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:43.829022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:18:43.829022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:18:43.830024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:43.830024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:18:43.830024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:43.830024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:18:43.830024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:18:43.830024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:43.830024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:18:43.830024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.830024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:18:43.830024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.830024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:43.830024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:43.830024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:43.830024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:18:43.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:18:43.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:43.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:43.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:43.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:43.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:43.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:43.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:43.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:43.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:43.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:43.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:43.830872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:43.830872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:43.830872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:43.830872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:43.830872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:43.830872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:43.830872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:43.830872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:43.830872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:43.830872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.830872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:43.831651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.831651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:43.831651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:43.831651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:43.831651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:43.831651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:43.831651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.831651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:43.831651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:43.831651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.831651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:43.831651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:43.831651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.831651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:43.831651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:43.831651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.831651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:18:43.832149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:18:43.832149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:43.832149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:43.832149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:43.832149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:43.832149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:43.832149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:43.832149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:43.832149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:43.832149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:43.832149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:43.832149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:43.832149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:43.832149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:43.832149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:43.832149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:43.832149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:43.832149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:43.832149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:43.832640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:43.832640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:43.832640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.832640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:43.832841 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:43.833169 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:43.833295 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:43.833447 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:43.833861 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:43.834162 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:43.834372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.835014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:43.835014 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:43.835014 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:43.835014 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:43.835014 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:43.835014 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:43.835014 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.835014 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:43.835765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.835765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:43.835765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:43.835765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:43.835765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.835765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:18:43.835765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:43.835765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:43.835765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:43.835765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:43.835765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:43.835765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:43.836144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:43.836144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:43.836144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:43.836144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:43.836144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:43.836144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:43.836144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:43.836144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:43.836144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:43.836144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:43.836144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:43.836144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:43.836144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:43.836501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.836501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:43.836568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.836568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:43.836568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:43.836568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:43.836568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:43.836568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:43.836568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.836568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:43.836568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:43.836568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.836568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:43.836568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:43.836568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.836568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:43.836568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:43.836568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:43.837081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:18:43.837081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:43.837081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:43.837081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:43.837081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:43.837081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:43.837081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:43.837081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:43.837081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:43.837081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:43.837081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:43.837081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:43.837081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:43.837081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:43.837081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:43.837081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:43.837081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:43.837081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:43.837081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:43.837636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:43.837636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:43.837636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:48.014362 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:48.015299 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:48.015299 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:48.016630 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:48.053887 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:48.054203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.054890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:48.054890 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:48.054890 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:48.054890 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:48.054890 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:48.054890 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:48.054890 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.054890 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:48.055720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.055720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:48.055720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:48.055720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:48.055720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.055720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:18:48.055720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:48.055720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:48.055720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:48.055720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:48.055720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:48.055720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:48.056278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:48.056278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:48.056278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:48.056278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:48.056278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:48.056278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:48.056278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:48.056278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:48.056278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:48.056278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:48.056278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:48.056278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:48.056278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:48.056944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.056944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:48.056944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.056944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:48.056944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:48.056944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:48.056944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:48.056944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:48.056944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.056944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:48.056944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:48.056944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.056944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:48.056944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:48.056944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.057643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:48.057643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:48.057643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.057643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:18:48.057643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:48.057643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:48.057643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:48.057643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:48.057643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:48.057643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:48.057643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:48.057643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:48.057643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:48.057643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:48.057643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:48.057643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:48.057643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:48.057643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:48.057643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:48.058508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:48.058508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:48.058508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:48.058508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:48.058508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.058508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:48.772970 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:48.773385 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:48.773589 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:48.774588 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:48.774813 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:48.774932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.775585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:48.775585 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:48.775585 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:48.775585 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:48.775585 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:48.775585 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:48.775585 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.775585 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:48.777144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.777144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:48.777144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:48.777144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:48.777144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.777144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:18:48.777144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:48.777144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:48.777144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:48.777144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:48.777144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:48.777144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:48.777758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:48.777758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:48.777758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:48.777758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:48.777758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:48.777758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:48.777758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:48.777758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:48.777758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:48.777758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:48.777758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:48.777758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:48.777758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:48.778351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.778351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:48.778897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.778897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:48.778897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:48.778897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:48.778897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:48.778897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:48.778897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.778897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:48.778897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:48.778897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.778897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:48.778897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:48.778897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.778897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:48.778897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:48.778897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.779653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:18:48.779653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:48.779653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:48.779653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:48.779653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:48.779653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:48.779653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:48.779653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:48.779653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:48.779653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:48.779653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:48.779653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:48.779653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:48.779653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:48.779653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:48.779653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:48.779653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:48.779653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:48.779653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:48.780470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:48.780470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.780470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:48.815974 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:48.816328 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:48.816501 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:48.816716 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:18:48.817455 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:48.817864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.818782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:48.818782 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:48.818782 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:48.818782 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:48.818782 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:48.818782 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:48.818782 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.818782 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:48.820095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.820095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:18:48.820095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:48.820095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:48.820095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:48.820095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:48.820095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:48.820095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:48.820095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:48.820095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:48.820095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:48.820095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:18:48.820095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:48.820095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:18:48.820095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:18:48.821000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:48.821000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:18:48.821000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:48.821000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:18:48.821000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:18:48.821000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:48.821000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:18:48.821000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.821000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:18:48.821604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.821604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:48.821604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:48.821604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:48.821604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.821604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:18:48.821604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:18:48.821604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:18:48.821604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:48.821604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:48.821604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:48.821604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:48.822465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:48.822465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:48.822465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:48.822465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:48.822465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:48.822465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:48.822465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:48.822465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:48.822465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:48.822465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:48.822465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:48.822465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:48.822465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:48.822465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:48.823436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:48.823436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:48.823436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.823436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:48.823784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.823784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:48.823784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:48.823784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:48.823784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:48.823784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:48.823784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.823784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:48.823784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:48.823784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.823784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:48.823784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:48.823784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.823784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:48.823784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:48.823784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.824634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:18:48.824634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:18:48.824634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:18:48.824634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:48.824634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:48.824634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:48.824634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:48.824634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:48.824634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:48.824634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:48.824634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:48.824634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:48.824634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:48.824634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:48.824634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:48.824634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:48.824634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:48.824634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:48.825289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:48.825289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:48.825289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:48.825289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:48.825289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:48.825289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.825289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:48.825944 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:48.826399 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:48.826541 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:48.826782 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:48.827224 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:48.827540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.828103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:48.828103 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:48.828103 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:48.828103 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:48.828103 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:48.828103 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:48.828103 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.828103 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:48.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:18:48.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:48.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:48.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:48.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:48.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:48.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:48.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:48.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:48.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:48.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:18:48.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:48.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:18:48.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:18:48.829332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:48.829332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:18:48.829332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:48.829332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:18:48.829332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:18:48.829332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:48.829332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:18:48.829332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.829332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:18:48.829728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.829728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:48.829728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:48.829728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:48.829728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.829728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:18:48.829728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:18:48.829728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:48.829728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:48.829728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:48.829728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:48.830038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:48.830038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:48.830038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:48.830038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:48.830038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:48.830038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:48.830038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:48.830038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:48.830038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:48.830038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:48.830038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:48.830038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:48.830038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:48.830038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:48.830435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:48.830435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:48.830435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.830435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:48.830775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.830775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:48.830775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:48.830775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:48.830775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:48.830775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:48.830775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.830775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:48.830775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:48.830775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.830775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:48.830775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:48.830775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.830775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:48.830775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:48.830775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.830775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:18:48.831247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:18:48.831247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:48.831247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:48.831247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:48.831247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:48.831247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:48.831247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:48.831247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:48.831247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:48.831247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:48.831247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:48.831247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:48.831247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:48.831247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:48.831247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:48.831247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:48.831247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:48.831247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:48.831247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:48.831935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:48.831935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:48.831935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.831935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:48.833165 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:48.833420 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:48.833530 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:48.833687 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:48.834165 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:48.834560 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:48.834837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.835757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:48.835757 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:48.835757 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:48.835757 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:48.835757 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:48.835757 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:48.835757 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.835757 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:48.836693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.836693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:48.836693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:48.836693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:48.836693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.836693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:18:48.836693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:48.836693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:48.836693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:48.836693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:48.836693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:48.836693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:48.836960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:48.836960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:48.836960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:48.836960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:48.836960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:48.836960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:48.836960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:48.836960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:48.836960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:48.836960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:48.836960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:48.836960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:48.836960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:48.837184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.837184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:48.839713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.839713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:48.839713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:48.839713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:48.839713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:48.839713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:48.839713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.839713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:48.839713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:48.839713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.839713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:48.839713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:48.839713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.839713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:48.839713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:48.839713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:48.840049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:18:48.840049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:48.840049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:48.840049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:48.840049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:48.840049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:48.840049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:48.840049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:48.840049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:48.840049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:48.840049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:48.840049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:48.840049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:48.840049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:48.840049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:48.840049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:48.840049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:48.840049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:48.840049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:48.840383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:48.840383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:48.840383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:53.015095 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:53.016258 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:53.016258 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:53.017375 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:53.055521 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:53.055972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.056454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:53.056454 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:53.056454 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:53.056454 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:53.056454 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:53.056454 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:53.056454 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.056454 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:53.057281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.057281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:53.057281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:53.057281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:53.057281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.057281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:18:53.057281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:53.057281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:53.057281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:53.057281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:53.057281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:53.057281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:53.058616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:53.058616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:53.058616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:53.058616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:53.058616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:53.058616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:53.058616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:53.058616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:53.058616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:53.058616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:53.058616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:53.058616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:53.058616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:53.059448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.059448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:53.059448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.059448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:53.059448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:53.059448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:53.059448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:53.059448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:53.059448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.059448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:53.059448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:53.059448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.059448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:53.059448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:53.059448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.060302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:53.060302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:53.060302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.060302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:18:53.060302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:53.060302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:53.060302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:53.060302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:53.060302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:53.060302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:53.060302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:53.060302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:53.060302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:53.060302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:53.060302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:53.060302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:53.060302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:53.060302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:53.060302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:53.061217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:53.061217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:53.061217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:53.061217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:53.061217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.061217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:53.773444 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:53.774106 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:53.774315 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:53.775593 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:53.775849 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:53.776147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.777070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:53.777070 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:53.777070 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:53.777070 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:53.777070 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:53.777070 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:53.777070 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.777070 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:53.778623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.778623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:53.778623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:53.778623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:53.778623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.778623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:18:53.778623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:53.778623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:53.778623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:53.778623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:53.778623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:53.778623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:53.779070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:53.779070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:53.779070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:53.779070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:53.779070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:53.779070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:53.779070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:53.779070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:53.779070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:53.779070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:53.779070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:53.779070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:53.779070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:53.779440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.779440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:53.780459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.780459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:53.780459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:53.780459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:53.780459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:53.780459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:53.780459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.780459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:53.780459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:53.780459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.780459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:53.780459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:53.780459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.780459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:53.780459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:53.780459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:18:53.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:53.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:53.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:53.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:53.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:53.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:53.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:53.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:53.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:53.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:53.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:53.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:53.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:53.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:53.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:53.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:53.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:53.780969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:53.781491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:53.781491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.781491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:53.816482 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:53.816998 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:53.817267 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:53.817552 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:18:53.818372 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:53.818789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.819669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:53.819669 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:53.819669 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:53.819669 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:53.819669 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:53.819669 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:53.819669 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.819669 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:53.820931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.820931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:18:53.820931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:53.820931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:53.820931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:53.820931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:53.820931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:53.820931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:53.820931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:53.820931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:53.820931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:53.820931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:18:53.820931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:53.820931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:18:53.820931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:18:53.824590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:53.824590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:18:53.824590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:53.824590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:18:53.824590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:18:53.824590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:53.824590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:18:53.824590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.824590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:18:53.824590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.824590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:53.824590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:53.824590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:53.824590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.826049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:18:53.826049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:18:53.826049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:18:53.826049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:53.826049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:53.826049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:53.826049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:53.826049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:53.826049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:53.826049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:53.826049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:53.826049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:53.826049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:53.826611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:53.826611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:53.826611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:53.826611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:53.826611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:53.826611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:53.826611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:53.826611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:53.826611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:53.826611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:53.826611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.826611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:53.827407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.827407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:53.827407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:53.827407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:53.827407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:53.827407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:53.827407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.827407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:53.827407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:53.827407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.827407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:53.827407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:53.827407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.827407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:53.827407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:53.827407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.828503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:18:53.828503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:18:53.828503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:18:53.828503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:53.828503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:53.828503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:53.828503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:53.828503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:53.828503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:53.828503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:53.828503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:53.828503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:53.828503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:53.828503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:53.828503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:53.828503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:53.828503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:53.828503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:53.829251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:53.829251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:53.829251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:53.829251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:53.829251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:53.829251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.829251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:53.829251 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:53.829251 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:53.829251 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:53.829548 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:53.829548 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:53.829548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.829548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:53.829548 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:53.829548 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:53.829548 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:53.829548 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:53.829548 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:53.829548 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.829548 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:53.829548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.829809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:18:53.829809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:53.829809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:53.829809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:53.829809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:53.829809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:53.829809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:53.829809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:53.829809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:53.829809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:53.829809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:18:53.829809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:53.829809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:18:53.829809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:18:53.829809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:53.829809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:18:53.829809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:53.830167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:18:53.830167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:18:53.830167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:53.830167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:18:53.830167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.830167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:18:53.830377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.830377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:53.830377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:53.830377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:53.830377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.830377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:18:53.830377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:18:53.830377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:53.830377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:53.830377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:53.830377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:53.830724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:53.830724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:53.830724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:53.830724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:53.830724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:53.830724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:53.830724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:53.830724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:53.830724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:53.830724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:53.830724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:53.830724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:53.830724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:53.830724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:53.831102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:53.831102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:53.831102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.831102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:53.831596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.831596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:53.831596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:53.831596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:53.831596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:53.831596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:53.831596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.831596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:53.831596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:53.831596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.831596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:53.831596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:53.831596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.831596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:53.831596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:53.831596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.831596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:18:53.832023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:18:53.832023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:53.832023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:53.832023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:53.832023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:53.832023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:53.832023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:53.832023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:53.832023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:53.832023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:53.832023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:53.832023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:53.832023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:53.832023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:53.832023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:53.832023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:53.832023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:53.832023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:53.832023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:53.832398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:53.832398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:53.832398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.832398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:53.833469 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:53.833670 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:53.833805 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:53.834037 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:53.834418 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:53.834660 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:53.834837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.835406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:53.835406 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:53.835406 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:53.835406 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:53.835406 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:53.835406 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:53.835406 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.835406 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:53.836515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.836515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:53.836515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:53.836515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:53.836515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.836515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:18:53.836515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:53.836515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:53.836515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:53.836515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:53.836515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:53.836515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:53.836805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:53.836805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:53.836805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:53.836805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:53.836805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:53.836805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:53.836805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:53.836805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:53.836805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:53.836805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:53.836805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:53.836805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:53.836805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:53.837044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.837044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:53.837540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.837540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:53.837540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:53.837540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:53.837540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:53.837540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:53.837540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.837540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:53.837540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:53.837540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.837540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:53.837540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:53.837540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.837540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:53.837540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:53.837540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:53.837870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:18:53.837870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:53.837870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:53.837870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:53.837870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:53.837870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:53.837870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:53.837870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:53.837870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:53.837870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:53.837870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:53.837870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:53.837870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:53.837870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:53.837870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:53.837870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:53.837870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:53.837870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:53.837870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:53.838242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:53.838242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:53.838242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:58.015842 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:58.016364 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:58.016555 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:58.017798 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:58.053161 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:58.053439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.054073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:58.054073 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:58.054073 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:58.054073 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:58.054073 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:58.054073 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:58.054073 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.054073 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:58.055924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.055924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:58.055924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:58.055924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:58.055924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.055924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:18:58.055924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:58.055924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:58.055924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:58.055924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:58.055924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:58.055924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:58.056274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:58.056274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:58.056274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:58.056274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:58.056274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:58.056274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:58.056274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:58.056274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:58.056274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:58.056274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:58.056274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:58.056274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:58.056274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:58.056551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.056551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:58.057410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.057410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:58.057410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:58.057410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:58.057410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:58.057410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:58.057410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.057410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:58.057410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:58.057410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.057410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:58.057410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:58.057410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.057410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:58.057410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:58.057410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.057802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:18:58.057802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:58.057802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:58.057802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:58.057802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:58.057802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:58.057802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:58.057802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:58.057802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:58.057802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:58.057802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:58.057802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:58.057802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:58.057802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:58.057802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:58.057802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:58.057802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:58.057802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:58.057802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:58.058213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:58.058213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.058213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:58.774120 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:58.774955 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:58.775186 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:58.777694 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:58.778092 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:58.778309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.779277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:58.779277 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:58.779277 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:58.779277 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:58.779277 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:58.779277 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:58.779277 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.779277 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:58.781294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.781294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:58.781294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:58.781294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:58.781294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.781294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:18:58.781294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:58.781294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:58.781294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:58.781294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:58.781294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:58.781294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:58.781906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:58.781906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:58.781906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:58.781906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:58.781906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:58.781906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:58.781906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:58.781906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:58.781906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:58.781906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:58.781906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:58.781906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:58.781906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:58.782459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.782459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:58.782459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.782459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:58.782459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:58.782459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:58.782459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:58.782459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:58.782459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.782459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:58.782459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:58.782459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.782459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:58.782459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:58.782459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.783127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:58.783127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:58.783127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.783127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:18:58.783127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:58.783127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:58.783127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:58.783127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:58.783127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:58.783127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:58.783127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:58.783127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:58.783127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:58.783127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:58.783127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:58.783127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:58.783127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:58.783127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:58.783127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:58.783945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:58.783945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:58.783945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:58.783945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:58.783945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.783945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:58.817146 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:58.817555 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:58.817713 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:58.817996 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:18:58.818918 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:58.819274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.820290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:58.820290 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:58.820290 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:58.820290 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:58.820290 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:58.820290 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:58.820290 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.820290 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:58.821647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.821647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:18:58.821647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:58.821647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:58.821647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:58.821647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:58.821647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:58.821647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:58.821647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:58.821647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:58.821647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:58.821647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:18:58.821647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:58.821647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:18:58.821647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:18:58.822258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:58.822258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:18:58.822258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:58.822258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:18:58.822258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:18:58.822258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:58.822258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:18:58.822258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.822258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:18:58.823121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.823121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:58.823121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:58.823121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:58.823121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.823121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:18:58.823121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:18:58.823121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:18:58.823121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:58.823121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:58.823121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:58.823121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:58.824511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:58.824511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:58.824511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:58.824511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:58.824511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:58.824511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:58.824511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:58.824511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:58.824511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:58.824511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:58.824511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:58.824511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:58.824511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:58.824511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:58.825581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:58.825581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:58.825581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.825581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:58.825581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.825581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:58.825581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:58.825581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:58.825581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:58.825581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:58.825581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.825581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:58.825581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:58.825581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.826225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:58.826225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:58.826225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.826225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:58.826225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:58.826225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.826225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:18:58.826225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:18:58.826225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:18:58.826225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:58.826225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:58.826225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:58.826225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:58.826225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:58.826225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:58.826225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:58.826225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:58.826225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:58.826225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:58.828947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:58.828947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:58.828947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:58.828947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:58.828947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:58.828947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:58.828947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:58.828947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:58.828947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:58.828947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:58.828947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.828947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:58.828947 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:58.828947 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:58.829579 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:58.829579 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:58.829579 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:58.829579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.829579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:58.829579 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:58.829579 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:58.829579 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:58.829579 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:58.829579 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:58.829579 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.829579 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:58.830005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.830005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:18:58.830005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:58.830005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:58.830005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:58.830005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:58.830005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:58.830005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:58.830005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:58.830005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:18:58.830005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:58.830005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:18:58.830005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:58.830005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:18:58.830005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:18:58.830420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:58.830420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:18:58.830420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:58.830420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:18:58.830420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:18:58.830420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:58.830420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:18:58.830420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.830420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:18:58.830670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.830670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:58.830670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:58.830670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:58.830670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.830670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:18:58.830670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:18:58.830670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:58.830670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:58.830670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:58.830670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:58.830987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:58.830987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:58.830987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:58.830987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:58.830987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:58.830987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:58.830987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:58.830987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:58.830987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:58.830987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:58.830987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:58.830987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:58.830987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:58.830987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:58.831387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:58.831387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:58.831387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.831387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:58.831767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.831767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:58.831767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:58.831767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:58.831767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:58.831767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:58.831767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.831767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:58.831767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:58.831767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.831767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:58.831767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:58.831767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.831767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:58.831767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:58.831767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.831767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:18:58.832724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:18:58.832724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:18:58.832724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:18:58.832724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:58.832724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:58.832724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:58.832724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:58.832724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:58.832724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:58.832724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:58.832724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:58.832724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:58.832724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:58.832724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:58.832724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:58.832724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:58.832724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:58.832724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:58.832724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:58.834396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:58.834396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:58.834396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.834396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:18:58.834396 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:18:58.834396 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:18:58.834396 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:18:58.834396 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:18:58.834837 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:58.835217 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:18:58.835454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.836431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:18:58.836431 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:58.836431 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:18:58.836431 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:18:58.836431 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:58.836431 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:18:58.836431 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.836431 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:18:58.837408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.837408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:18:58.837408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:58.837408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:18:58.837408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.837408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:18:58.837408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:58.837408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:58.837408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:58.837408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:58.837408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:58.837408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:58.837779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:18:58.837779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:58.837779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:18:58.837779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:58.837779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:18:58.837779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:18:58.837779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:58.837779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:18:58.837779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:58.837779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:18:58.837779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:18:58.837779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:58.837779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:18:58.838121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.838121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:18:58.838341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.838341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:18:58.838341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:18:58.838341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:18:58.838341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:18:58.838341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:18:58.838341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.838341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:18:58.838341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:18:58.838341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.838341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:18:58.838341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:18:58.838341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.838341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:18:58.838341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:18:58.838341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:18:58.838821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:18:58.838821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:18:58.838821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:58.838821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:58.838821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:18:58.838821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:18:58.838821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:18:58.838821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:18:58.838821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:18:58.838821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:18:58.838821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:18:58.838821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:18:58.838821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:18:58.838821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:18:58.838821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:18:58.838821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:18:58.838821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:18:58.838821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:18:58.838821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:18:58.839380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:18:58.839380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:18:58.839380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:03.016519 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:03.017233 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:03.017349 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:03.018570 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:03.025643 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=240 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:19:03.054658 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:03.055111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.055921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:03.055921 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:03.055921 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:03.055921 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:03.055921 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:03.055921 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:03.055921 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.055921 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:03.056592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.056592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:03.056592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:03.056592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:03.056592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.056592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:19:03.056592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:03.056592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:03.056592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:03.056592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:03.056592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:03.056592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:03.057215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:03.057215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:03.057215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:03.057215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:03.057215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:03.057215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:03.057215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:03.057215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:03.057215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:03.057215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:03.057215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:03.057215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:03.057215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:03.057857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.057857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:03.057857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.057857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:03.057857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:03.057857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:03.057857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:03.057857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:03.057857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.057857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:03.057857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:03.057857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.057857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:03.057857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:03.057857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.058598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:03.058598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:03.058598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.058598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:19:03.058598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:03.058598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:03.058598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:03.058598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:03.058598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:03.058598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:03.058598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:03.058598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:03.058598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:03.058598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:03.058598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:03.058598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:03.058598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:03.058598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:03.058598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:03.059536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:03.059536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:03.059536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:03.059536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:03.059536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.059536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:03.774784 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:03.775432 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:03.775637 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:03.776872 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:03.777143 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:03.777310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.778081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:03.778081 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:03.778081 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:03.778081 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:03.778081 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:03.778081 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:03.778081 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.778081 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:03.779998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.779998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:03.779998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:03.779998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:03.779998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.779998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:19:03.779998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:03.779998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:03.779998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:03.779998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:03.779998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:03.779998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:03.780721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:03.780721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:03.780721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:03.780721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:03.780721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:03.780721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:03.780721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:03.780721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:03.780721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:03.780721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:03.780721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:03.780721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:03.780721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:03.781470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.781470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:03.781470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.781470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:03.781470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:03.781470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:03.781470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:03.781470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:03.781470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.781470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:03.781470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:03.781470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.781470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:03.781470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:03.781470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.782229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:03.782229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:03.782229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.782229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:19:03.782229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:03.782229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:03.782229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:03.782229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:03.782229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:03.782229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:03.782229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:03.782229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:03.782229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:03.782229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:03.782229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:03.782229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:03.782229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:03.782229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:03.782229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:03.783170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:03.783170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:03.783170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:03.783170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:03.783170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.783170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:03.817625 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:03.818117 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:03.818328 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:03.818587 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:19:03.819365 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:03.819628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.820368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:03.820368 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:03.820368 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:03.820368 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:03.820368 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:03.820368 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:03.820368 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.820368 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:03.821588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.821588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:19:03.821588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:03.821588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:03.821588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:03.821588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:03.821588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:03.821588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:03.821588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:03.821588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:03.821588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:03.821588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:19:03.821588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:03.821588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:19:03.821588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:19:03.822381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:03.822381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:19:03.822381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:03.822381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:19:03.822381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:19:03.822381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:03.822381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:19:03.822381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.822381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:19:03.823030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.823030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:03.823030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:03.823030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:03.823030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.823030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:19:03.823030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:19:03.823030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:19:03.823030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:03.823030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:03.823030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:03.823030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:03.823665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:03.823665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:03.823665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:03.823665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:03.823665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:03.823665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:03.823665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:03.823665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:03.823665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:03.823665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:03.823665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:03.823665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:03.823665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:03.823665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:03.824302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:03.824302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:03.824302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.824302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:03.824688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.824688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:03.824688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:03.824688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:03.824688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:03.824688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:03.824688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.824688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:03.824688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:03.824688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.824688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:03.824688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:03.824688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.824688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:03.824688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:03.824688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.825393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:19:03.825393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:19:03.825393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:19:03.825393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:03.825393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:03.825393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:03.825393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:03.825393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:03.825393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:03.825393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:03.825393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:03.825393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:03.825393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:03.825393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:03.825393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:03.825393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:03.825393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:03.825393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:03.826139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:03.826139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:03.826139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:03.826139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:03.826139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:03.826139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.826139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:03.827317 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:03.827824 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:03.828028 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:03.828236 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:03.828810 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:03.829129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.829869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:03.829869 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:03.829869 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:03.829869 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:03.829869 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:03.829869 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:03.829869 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.829869 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:03.831464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.831464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:19:03.831464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:03.831464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:03.831464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:03.831464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:03.831464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:03.831464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:03.831464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:03.831464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:03.831464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:03.831464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:19:03.831464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:03.831464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:19:03.831464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:19:03.831908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:03.831908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:19:03.831908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:03.831908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:19:03.831908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:19:03.831908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:03.831908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:19:03.831908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.831908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:19:03.833021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.833021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:03.833021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:03.833021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:03.833021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.833021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:19:03.833021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:19:03.833021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:03.833021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:03.833021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:03.833021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:03.833388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:03.833388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:03.833388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:03.833388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:03.833388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:03.833388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:03.833388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:03.833388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:03.833388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:03.833388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:03.833388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:03.833388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:03.833388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:03.833388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:03.833772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:03.833772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:03.833772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.833772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:03.834858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.834858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:03.834858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:03.834858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:03.834858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:03.834858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:03.834858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.834858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:03.834858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:03.834858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.834858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:03.834858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:03.834858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.834858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:03.834858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:03.834858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.834858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:19:03.835362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:19:03.835362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:03.835362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:03.835362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:03.835362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:03.835362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:03.835362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:03.835362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:03.835362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:03.835362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:03.835362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:03.835362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:03.835362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:03.835362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:03.835362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:03.835362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:03.835362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:03.835362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:03.835362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:03.835862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:03.835862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:03.835862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.835862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:03.835862 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:03.835862 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:03.836125 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:03.836178 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:03.836506 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:03.836776 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:03.836932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.837422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:03.837422 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:03.837422 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:03.837422 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:03.837422 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:03.837422 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:03.837422 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.837422 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:03.838029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.838029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:03.838029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:03.838029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:03.838029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.838029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:19:03.838029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:03.838029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:03.838029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:03.838029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:03.838029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:03.838029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:03.838551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:03.838551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:03.838551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:03.838551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:03.838551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:03.838551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:03.838551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:03.838551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:03.838551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:03.838551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:03.838551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:03.838551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:03.838551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:03.838946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.838946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:03.838946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.838946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:03.838946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:03.838946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:03.838946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:03.838946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:03.838946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.838946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:03.838946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:03.838946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.838946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:03.838946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:03.838946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.839453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:03.839453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:03.839453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:03.839453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:19:03.839453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:03.839453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:03.839453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:03.839453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:03.839453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:03.839453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:03.839453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:03.839453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:03.839453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:03.839453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:03.839453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:03.839453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:03.839453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:03.839453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:03.839453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:03.840093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:03.840093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:03.840093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:03.840093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:03.840093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:03.840093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:08.017283 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:08.018069 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:08.018069 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:08.018990 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:08.054162 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:08.054603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.055476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:08.055476 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:08.055476 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:08.055476 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:08.055476 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:08.055476 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:08.055476 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.055476 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:08.056358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.056358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:08.056358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:08.056358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:08.056358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.056358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:19:08.056358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:08.056358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:08.056358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:08.056358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:08.056358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:08.056358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:08.056961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:08.056961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:08.056961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:08.056961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:08.056961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:08.056961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:08.056961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:08.056961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:08.056961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:08.056961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:08.056961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:08.056961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:08.056961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:08.057537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.057537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:08.057537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.057537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:08.057537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:08.057537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:08.057537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:08.057537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:08.057537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.057537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:08.057537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:08.057537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.057537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:08.057537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:08.057537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.058211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:08.058211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:08.058211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.058211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:19:08.058211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:08.058211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:08.058211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:08.058211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:08.058211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:08.058211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:08.058211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:08.058211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:08.058211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:08.058211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:08.058211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:08.058211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:08.058211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:08.058211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:08.058211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:08.059032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:08.059032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:08.059032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:08.059032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:08.059032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.059032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:08.775511 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:08.776149 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:08.776355 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:08.777558 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:08.777993 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:08.778250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.779794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:08.779794 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:08.779794 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:08.779794 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:08.779794 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:08.779794 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:08.779794 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.779794 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:08.781376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.781376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:08.781376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:08.781376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:08.781376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.781376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:19:08.781376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:08.781376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:08.781376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:08.781376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:08.781376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:08.781376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:08.782287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:08.782287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:08.782287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:08.782287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:08.782287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:08.782287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:08.782287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:08.782287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:08.782287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:08.782287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:08.782287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:08.782287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:08.782287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:08.783109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.783109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:08.783109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.783109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:08.783109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:08.783109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:08.783109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:08.783109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:08.783109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.783109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:08.783109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:08.783109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.783109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:08.783109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:08.783109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.784086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:08.784086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:08.784086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.784086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:19:08.784086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:08.784086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:08.784086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:08.784086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:08.784086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:08.784086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:08.784086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:08.784086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:08.784086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:08.784086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:08.784086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:08.784086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:08.784086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:08.784086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:08.784086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:08.785300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:08.785300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:08.785300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:08.785300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:08.785300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.785300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:08.818189 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:08.818622 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:08.818831 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:08.819010 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:19:08.819797 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:08.820131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.820985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:08.820985 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:08.820985 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:08.820985 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:08.820985 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:08.820985 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:08.820985 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.820985 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:08.822172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.822172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:19:08.822172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:08.822172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:08.822172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:08.822172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:08.822172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:08.822172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:08.822172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:08.822172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:08.822172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:08.822172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:19:08.822172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:08.822172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:19:08.822172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:19:08.823082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:08.823082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:19:08.823082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:08.823082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:19:08.823082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:19:08.823082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:08.823082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:19:08.823082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.823082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:19:08.823655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.823655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:08.823655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:08.823655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:08.823655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.823655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:19:08.823655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:19:08.823655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:19:08.823655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:08.823655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:08.823655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:08.823655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:08.824581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:08.824581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:08.824581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:08.824581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:08.824581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:08.824581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:08.824581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:08.824581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:08.824581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:08.824581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:08.824581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:08.824581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:08.824581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:08.824581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:08.825434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:08.825434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:08.825434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.825434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:08.825434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.825434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:08.825434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:08.825434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:08.825434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:08.825434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:08.825434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.825434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:08.825434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:08.825434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.826255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:08.826255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:08.826255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.826255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:08.826255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:08.826255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.826255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:19:08.826255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:19:08.826255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:19:08.826255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:08.826255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:08.826255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:08.826255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:08.826255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:08.826255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:08.826255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:08.826255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:08.826255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:08.826255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:08.827065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:08.827065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:08.827065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:08.827065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:08.827065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:08.827065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:08.827065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:08.827065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:08.827065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:08.827065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:08.827065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.827065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:08.827842 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:08.828230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:08.828398 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:08.828592 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:08.829000 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:08.829247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.829847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:08.829847 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:08.829847 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:08.829847 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:08.829847 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:08.829847 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:08.829847 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.829847 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:08.830646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.830646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:19:08.830646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:08.830646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:08.830646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:08.830646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:08.830646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:08.830646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:08.830646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:08.830646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:08.830646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:08.830646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:19:08.830646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:08.830646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:19:08.830646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:19:08.831061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:08.831061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:19:08.831061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:08.831061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:19:08.831061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:19:08.831061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:08.831061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:19:08.831061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.831061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:19:08.831490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.831490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:08.831490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:08.831490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:08.831490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.831490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:19:08.831490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:19:08.831490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:08.831490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:08.831490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:08.831490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:08.831827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:08.831827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:08.831827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:08.831827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:08.831827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:08.831827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:08.831827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:08.831827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:08.831827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:08.831827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:08.831827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:08.831827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:08.831827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:08.831827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:08.832226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:08.832226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:08.832226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.832226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:08.832546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.832546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:08.832546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:08.832546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:08.832546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:08.832546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:08.832546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.832546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:08.832546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:08.832546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.832546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:08.832546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:08.832546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.832546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:08.832546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:08.832546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.832546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:19:08.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:19:08.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:08.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:08.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:08.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:08.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:08.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:08.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:08.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:08.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:08.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:08.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:08.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:08.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:08.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:08.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:08.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:08.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:08.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:08.833763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:08.833763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:08.833763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.833763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:08.835625 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:08.835875 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:08.835971 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:08.836127 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:08.836523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:08.836815 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:08.836973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.838126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:08.838126 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:08.838126 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:08.838126 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:08.838126 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:08.838126 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:08.838126 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.838126 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:08.838394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.838394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:08.838394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:08.838394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:08.838394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.838394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:19:08.838394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:08.838394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:08.838394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:08.838394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:08.838394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:08.838394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:08.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:08.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:08.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:08.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:08.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:08.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:08.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:08.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:08.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:08.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:08.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:08.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:08.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:08.839158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.839158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:08.839335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.839335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:08.839335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:08.839335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:08.839335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:08.839335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:08.839335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.839335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:08.839335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:08.839335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.839335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:08.839335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:08.839335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.839335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:08.839335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:08.839335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:08.839820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:19:08.839820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:08.839820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:08.839820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:08.839820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:08.839820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:08.839820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:08.839820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:08.839820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:08.839820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:08.839820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:08.839820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:08.839820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:08.839820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:08.839820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:08.839820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:08.839820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:08.839820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:08.839820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:08.840515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:08.840515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:08.840515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:13.017994 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:13.018720 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:13.018879 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:13.020291 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:13.050026 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:13.050379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.051121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:13.051121 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:13.051121 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:13.051121 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:13.051121 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:13.051121 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:13.051121 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.051121 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:13.051795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.051795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:13.051795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:13.051795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:13.051795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.051795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:19:13.051795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:13.051795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:13.051795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:13.051795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:13.051795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:13.051795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:13.052661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:13.052661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:13.052661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:13.052661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:13.052661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:13.052661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:13.052661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:13.052661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:13.052661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:13.052661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:13.052661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:13.052661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:13.052661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:13.053549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.053549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:13.053549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.053549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:13.053549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:13.053549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:13.053549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:13.053549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:13.053549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.053549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:13.053549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:13.053549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.053549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:13.053549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:13.053549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.054585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:13.054585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:13.054585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.054585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:19:13.054585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:13.054585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:13.054585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:13.054585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:13.054585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:13.054585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:13.054585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:13.054585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:13.054585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:13.054585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:13.054585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:13.054585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:13.054585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:13.054585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:13.054585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:13.055901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:13.055901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:13.055901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:13.055901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:13.055901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.055901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:13.776167 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:13.776832 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:13.777036 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:13.778229 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:13.778601 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:13.778861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.779854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:13.779854 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:13.779854 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:13.779854 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:13.779854 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:13.779854 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:13.779854 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.779854 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:13.781411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.781411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:13.781411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:13.781411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:13.781411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.781411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:19:13.781411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:13.781411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:13.781411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:13.781411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:13.781411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:13.781411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:13.781734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:13.781734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:13.781734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:13.781734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:13.781734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:13.781734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:13.781734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:13.781734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:13.781734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:13.781734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:13.781734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:13.781734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:13.781734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:13.782025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.782025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:13.783208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.783208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:13.783208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:13.783208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:13.783208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:13.783208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:13.783208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.783208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:13.783208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:13.783208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.783208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:13.783208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:13.783208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.783208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:13.783208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:13.783208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.783637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:19:13.783637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:13.783637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:13.783637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:13.783637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:13.783637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:13.783637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:13.783637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:13.783637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:13.783637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:13.783637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:13.783637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:13.783637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:13.783637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:13.783637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:13.783637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:13.783637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:13.783637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:13.783637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:13.784204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:13.784204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.784204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:13.818708 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:13.819860 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:13.819860 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:13.819860 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:19:13.820183 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:13.820480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.821491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:13.821491 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:13.821491 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:13.821491 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:13.821491 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:13.821491 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:13.821491 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.821491 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:13.822931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.822931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:19:13.822931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:13.822931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:13.822931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:13.822931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:13.822931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:13.822931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:13.822931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:13.822931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:13.822931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:13.822931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:19:13.822931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:13.822931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:19:13.822931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:19:13.823819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:13.823819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:19:13.823819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:13.823819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:19:13.823819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:19:13.823819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:13.823819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:19:13.823819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.823819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:19:13.824440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.824440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:13.824440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:13.824440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:13.824440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.824440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:19:13.824440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:19:13.824440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:19:13.824440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:13.824440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:13.824440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:13.824440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:13.825487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:13.825487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:13.825487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:13.825487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:13.825487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:13.825487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:13.825487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:13.825487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:13.825487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:13.825487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:13.825487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:13.825487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:13.825487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:13.825487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:13.826151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:13.826151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:13.826151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.826151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:13.826652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.826652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:13.826652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:13.826652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:13.826652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:13.826652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:13.826652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.826652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:13.826652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:13.826652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.826652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:13.826652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:13.826652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.826652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:13.826652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:13.826652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.827374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:19:13.827374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:19:13.827374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:19:13.827374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:13.827374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:13.827374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:13.827374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:13.827374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:13.827374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:13.827374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:13.827374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:13.827374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:13.827374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:13.827374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:13.827374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:13.827374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:13.827374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:13.827374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:13.828193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:13.828193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:13.828193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:13.828193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:13.828193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:13.828193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.828193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:13.828193 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:13.829898 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:13.830160 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:13.830423 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:13.830958 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:13.831300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.832228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:13.832228 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:13.832228 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:13.832228 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:13.832228 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:13.832228 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:13.832228 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.832228 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:13.833584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.833584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:19:13.833584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:13.833584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:13.833584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:13.833584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:13.833584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:13.833584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:13.833584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:13.833584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:13.833584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:13.833584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:19:13.833584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:13.833584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:19:13.833584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:19:13.833973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:13.833973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:19:13.833973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:13.833973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:19:13.833973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:19:13.833973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:13.833973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:19:13.833973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.833973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:19:13.835145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.835145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:13.835145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:13.835145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:13.835145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.835145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:19:13.835145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:19:13.835145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:13.835145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:13.835145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:13.835145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:13.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:13.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:13.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:13.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:13.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:13.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:13.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:13.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:13.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:13.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:13.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:13.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:13.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:13.835487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:13.835914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:13.835914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:13.835914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.835914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:13.837201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.837201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:13.837201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:13.837201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:13.837201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:13.837201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:13.837201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.837201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:13.837201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:13.837201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.837201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:13.837201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:13.837201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.837201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:13.837201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:13.837201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.837201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:19:13.837582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:19:13.837582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:13.837582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:13.837582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:13.837582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:13.837582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:13.837582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:13.837582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:13.837582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:13.837582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:13.837582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:13.837582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:13.837582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:13.837582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:13.837582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:13.837582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:13.837582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:13.837582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:13.837582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:13.838049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:13.838049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:13.838049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.838049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:13.838049 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:13.838049 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:13.838049 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:13.838049 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:13.838307 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:13.838513 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:13.838649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.839077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:13.839077 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:13.839077 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:13.839077 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:13.839077 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:13.839077 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:13.839077 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.839077 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:13.839848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.839848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:13.839848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:13.839848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:13.839848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.839848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:19:13.839848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:13.839848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:13.839848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:13.839848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:13.839848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:13.839848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:13.840104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:13.840104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:13.840104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:13.840104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:13.840104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:13.840104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:13.840104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:13.840104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:13.840104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:13.840104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:13.840104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:13.840104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:13.840104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:13.840351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.840351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:13.840849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.840849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:13.840849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:13.840849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:13.840849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:13.840849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:13.840849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.840849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:13.840849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:13.840849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.840849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:13.840849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:13.840849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.840849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:13.840849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:13.840849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:13.841222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:19:13.841222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:13.841222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:13.841222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:13.841222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:13.841222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:13.841222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:13.841222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:13.841222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:13.841222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:13.841222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:13.841222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:13.841222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:13.841222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:13.841222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:13.841222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:13.841222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:13.841222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:13.841222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:13.841783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:13.841783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:13.841783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:18.018531 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:18.019211 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:18.019211 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:18.020436 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:18.054483 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:18.054906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.055340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:18.055340 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:18.055340 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:18.055340 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:18.055340 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:18.055340 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:18.055340 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.055340 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:18.056021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.056021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:18.056021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:18.056021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:18.056021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.056021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:19:18.056021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:18.056021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:18.056021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:18.056021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:18.056021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:18.056021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:18.056371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:18.056371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:18.056371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:18.056371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:18.056371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:18.056371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:18.056371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:18.056371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:18.056371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:18.056371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:18.056371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:18.056371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:18.056371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:18.056650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.056650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:18.056906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.056906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:18.056906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:18.056906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:18.056906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:18.056906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:18.056906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.056906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:18.056906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:18.056906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.056906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:18.056906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:18.056906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.056906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:18.056906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:18.056906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.057877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:19:18.057877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:18.057877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:18.057877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:18.057877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:18.057877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:18.057877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:18.057877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:18.057877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:18.057877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:18.057877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:18.057877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:18.057877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:18.057877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:18.057877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:18.057877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:18.057877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:18.057877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:18.057877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:18.058502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:18.058502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.058502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:18.776779 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:18.777372 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:18.777550 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:18.778502 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:18.778799 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:18.779027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.779900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:18.779900 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:18.779900 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:18.779900 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:18.779900 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:18.779900 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:18.779900 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.779900 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:18.781334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.781334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:18.781334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:18.781334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:18.781334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.781334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:19:18.781334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:18.781334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:18.781334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:18.781334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:18.781334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:18.781334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:18.781994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:18.781994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:18.781994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:18.781994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:18.781994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:18.781994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:18.781994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:18.781994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:18.781994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:18.781994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:18.781994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:18.781994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:18.781994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:18.782654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.782654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:18.783317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.783317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:18.783317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:18.783317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:18.783317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:18.783317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:18.783317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.783317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:18.783317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:18.783317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.783317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:18.783317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:18.783317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.783317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:18.783317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:18.783317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.785008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:19:18.785008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:18.785008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:18.785008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:18.785008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:18.785008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:18.785008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:18.785008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:18.785008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:18.785008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:18.785008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:18.785008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:18.785008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:18.785008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:18.785008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:18.785008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:18.785008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:18.785008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:18.785008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:18.785851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:18.785851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.785851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:18.819351 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:18.819761 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:18.819920 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:18.820110 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:19:18.820489 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:18.820797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.821537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:18.821537 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:18.821537 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:18.821537 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:18.821537 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:18.821537 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:18.821537 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.821537 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:18.822666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.822666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:19:18.822666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:18.822666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:18.822666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:18.822666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:18.822666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:18.822666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:18.822666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:18.822666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:18.822666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:18.822666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:19:18.822666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:18.822666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:19:18.822666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:19:18.823357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:18.823357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:19:18.823357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:18.823357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:19:18.823357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:19:18.823357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:18.823357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:19:18.823357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.823357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:19:18.824109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.824109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:18.824109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:18.824109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:18.824109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.824109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:19:18.824109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:19:18.824109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:19:18.824109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:18.824109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:18.824109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:18.824109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:18.825251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:18.825251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:18.825251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:18.825251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:18.825251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:18.825251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:18.825251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:18.825251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:18.825251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:18.825251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:18.825251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:18.825251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:18.825251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:18.825251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:18.826462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:18.826462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:18.826462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.826462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:18.826462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.826462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:18.826462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:18.826462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:18.826462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:18.826462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:18.826462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.826462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:18.826462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:18.826462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.827376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:18.827376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:18.827376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.827376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:18.827376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:18.827376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.827376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:19:18.827376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:19:18.827376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:19:18.827376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:18.827376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:18.827376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:18.827376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:18.827376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:18.827376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:18.827376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:18.827376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:18.827376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:18.827376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:18.827866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:18.827866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:18.827866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:18.827866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:18.827866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:18.827866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:18.827866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:18.827866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:18.827866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:18.827866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:18.827866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.827866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:18.828821 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:18.829143 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:18.829262 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:18.829387 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:18.829621 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:18.829867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.830289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:18.830289 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:18.830289 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:18.830289 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:18.830289 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:18.830289 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:18.830289 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.830289 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:18.831032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.831032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:19:18.831032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:18.831032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:18.831032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:18.831032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:18.831032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:18.831032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:18.831032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:18.831032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:18.831032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:18.831032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:19:18.831032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:18.831032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:19:18.831032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:19:18.831449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:18.831449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:19:18.831449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:18.831449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:19:18.831449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:19:18.831449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:18.831449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:19:18.831449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.831449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:19:18.832206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.832206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:18.832206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:18.832206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:18.832206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.832206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:19:18.832206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:19:18.832206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:18.832206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:18.832206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:18.832206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:18.832505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:18.832505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:18.832505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:18.832505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:18.832505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:18.832505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:18.832505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:18.832505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:18.832505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:18.832505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:18.832505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:18.832505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:18.832505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:18.832505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:18.832870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:18.832870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:18.832870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.832870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:18.833591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.833591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:18.833591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:18.833591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:18.833591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:18.833591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:18.833591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.833591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:18.833591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:18.833591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.833591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:18.833591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:18.833591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.833591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:18.833591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:18.833591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.833591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:19:18.834058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:19:18.834058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:18.834058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:18.834058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:18.834058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:18.834058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:18.834058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:18.834058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:18.834058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:18.834058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:18.834058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:18.834058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:18.834058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:18.834058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:18.834058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:18.834058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:18.834058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:18.834058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:18.834058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:18.834552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:18.834552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:18.834552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.834552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:18.837941 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:18.838247 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:18.838395 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:18.838593 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:18.838865 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:18.839159 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:18.839368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.839924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:18.839924 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:18.839924 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:18.839924 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:18.839924 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:18.839924 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:18.839924 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.839924 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:18.840879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.840879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:18.840879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:18.840879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:18.840879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.840879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:19:18.840879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:18.840879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:18.840879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:18.840879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:18.840879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:18.840879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:18.841237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:18.841237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:18.841237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:18.841237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:18.841237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:18.841237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:18.841237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:18.841237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:18.841237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:18.841237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:18.841237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:18.841237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:18.841237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:18.841576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.841576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:18.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:18.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:18.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:18.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:18.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:18.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:18.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:18.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:18.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:18.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:18.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:18.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:18.842557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:19:18.842557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:18.842557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:18.842557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:18.842557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:18.842557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:18.842557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:18.842557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:18.842557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:18.842557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:18.842557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:18.842557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:18.842557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:18.842557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:18.842557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:18.842557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:18.842557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:18.842557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:18.842557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:18.843087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:18.843087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:18.843087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:23.019078 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:23.020109 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:23.020109 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:23.021284 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:23.051251 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:23.051403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.052254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:23.052254 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:23.052254 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:23.052254 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:23.052254 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:23.052254 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:23.052254 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.052254 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:23.054587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.054587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:23.054587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:23.054587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:23.054587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.054587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:19:23.054587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:23.054587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:23.054587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:23.054587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:23.054587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:23.054587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:23.054844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:23.054844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:23.054844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:23.054844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:23.054844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:23.054844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:23.054844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:23.054844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:23.054844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:23.054844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:23.054844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:23.054844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:23.054844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:23.055096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.055096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:23.056170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.056170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:23.056170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:23.056170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:23.056170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:23.056170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:23.056170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.056170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:23.056170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:23.056170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.056170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:23.056170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:23.056170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.056170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:23.056170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:23.056170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.056729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:19:23.056729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:23.056729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:23.056729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:23.056729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:23.056729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:23.056729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:23.056729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:23.056729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:23.056729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:23.056729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:23.056729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:23.056729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:23.056729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:23.056729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:23.056729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:23.056729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:23.056729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:23.056729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:23.057065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:23.057065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.057065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:23.777428 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:23.778158 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:23.778356 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:23.779625 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:23.780036 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:23.780321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.781343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:23.781343 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:23.781343 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:23.781343 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:23.781343 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:23.781343 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:23.781343 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.781343 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:23.782929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.782929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:23.782929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:23.782929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:23.782929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.782929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:19:23.782929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:23.782929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:23.782929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:23.782929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:23.782929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:23.782929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:23.783419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:23.783419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:23.783419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:23.783419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:23.783419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:23.783419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:23.783419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:23.783419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:23.783419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:23.783419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:23.783419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:23.783419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:23.783419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:23.783973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.783973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:23.783973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.783973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:23.783973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:23.783973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:23.783973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:23.783973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:23.783973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.783973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:23.783973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:23.783973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.783973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:23.783973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:23.783973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.784549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:23.784549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:23.784549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.784549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:19:23.784549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:23.784549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:23.784549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:23.784549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:23.784549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:23.784549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:23.784549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:23.784549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:23.784549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:23.784549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:23.784549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:23.784549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:23.784549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:23.784549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:23.784549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:23.785301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:23.785301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:23.785301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:23.785301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:23.785301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.785301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:23.819900 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:23.820365 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:23.820553 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:23.820777 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:19:23.821452 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:23.821738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.822487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:23.822487 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:23.822487 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:23.822487 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:23.822487 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:23.822487 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:23.822487 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.822487 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:23.823792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.823792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:19:23.823792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:23.823792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:23.823792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:23.823792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:23.823792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:23.823792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:23.823792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:23.823792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:23.823792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:23.823792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:19:23.823792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:23.823792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:19:23.823792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:19:23.824388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:23.824388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:19:23.824388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:23.824388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:19:23.824388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:19:23.824388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:23.824388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:19:23.824388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.824388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:19:23.826865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.826865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:23.826865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:23.826865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:23.826865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.826865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:19:23.826865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:19:23.826865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:19:23.826865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:23.826865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:23.826865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:23.826865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:23.827458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:23.827458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:23.827458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:23.827458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:23.827458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:23.827458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:23.827458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:23.827458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:23.827458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:23.827458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:23.827458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:23.827458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:23.827458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:23.827458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:23.828121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:23.828121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:23.828121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.828121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:23.829396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.829396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:23.829396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:23.829396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:23.829396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:23.829396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:23.829396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.829396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:23.829396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:23.829396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.829396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:23.829396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:23.829396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.829396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:23.829396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:23.829396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.830606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:19:23.830606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:19:23.830606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:19:23.830606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:23.830606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:23.830606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:23.830606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:23.830606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:23.830606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:23.830606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:23.830606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:23.830606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:23.830606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:23.830606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:23.830606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:23.830606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:23.830606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:23.830606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:23.831431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:23.831431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:23.831431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:23.831431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:23.831431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:23.831431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.831431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:23.831431 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:23.831431 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:23.831431 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:23.831961 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:23.831961 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:23.831961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.832173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:23.832173 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:23.832173 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:23.832173 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:23.832173 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:23.832173 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:23.832173 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.832173 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:23.833476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.833476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:19:23.833476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:23.833476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:23.833476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:23.833476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:23.833476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:23.833476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:23.833476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:23.833476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:23.833476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:23.833476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:19:23.833476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:23.833476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:19:23.833476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:19:23.834555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:23.834555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:19:23.834555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:23.834555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:19:23.834555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:19:23.834555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:23.834555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:19:23.834555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.834555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:19:23.835036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.835036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:23.835036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:23.835036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:23.835036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.835036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:19:23.835036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:19:23.835036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:23.835036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:23.835036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:23.835036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:23.836046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:23.836046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:23.836046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:23.836046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:23.836046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:23.836046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:23.836046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:23.836046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:23.836046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:23.836046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:23.836046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:23.836046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:23.836046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:23.836046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:23.836872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:23.836872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:23.836872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.836872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:23.836872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.836872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:23.836872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:23.836872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:23.836872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:23.836872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:23.836872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.836872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:23.836872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:23.836872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.837207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:23.837207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:23.837207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.837207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:23.837207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:23.837207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.837207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:19:23.837207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:19:23.837207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:23.837207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:23.837207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:23.837207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:23.837207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:23.837207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:23.837207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:23.837207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:23.837207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:23.837207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:23.837207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:23.837666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:23.837666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:23.837666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:23.837666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:23.837666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:23.837666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:23.837666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:23.837666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:23.837666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:23.837666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.837666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:23.838481 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:23.838795 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:23.838923 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:23.839077 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:23.839567 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:23.840000 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:23.840248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.841177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:23.841177 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:23.841177 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:23.841177 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:23.841177 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:23.841177 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:23.841177 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.841177 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:23.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:23.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:23.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:23.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:19:23.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:23.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:23.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:23.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:23.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:23.842121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:23.843066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:23.843066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:23.843066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:23.843066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:23.843066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:23.843066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:23.843066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:23.843066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:23.843066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:23.843066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:23.843066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:23.843066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:23.843066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:23.844005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.844005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:23.844005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.844005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:23.844005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:23.844005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:23.844005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:23.844005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:23.844005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.844005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:23.844005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:23.844005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.844005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:23.844005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:23.844005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.844649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:23.844649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:23.844649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:23.844649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:19:23.844649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:23.844649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:23.844649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:23.844649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:23.844649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:23.844649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:23.844649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:23.844649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:23.844649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:23.844649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:23.844649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:23.844649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:23.844649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:23.844649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:23.844649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:23.845117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:23.845117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:23.845117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:23.845117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:23.845117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:23.845117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:28.019714 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:28.020572 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:28.020844 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:28.022252 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:28.055422 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:28.055773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.056758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:28.056758 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:28.056758 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:28.056758 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:28.056758 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:28.056758 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:28.056758 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.056758 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:28.057861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.057861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:28.057861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:28.057861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:28.057861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.057861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:19:28.057861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:28.057861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:28.057861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:28.057861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:28.057861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:28.057861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:28.058266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:28.058266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:28.058266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:28.058266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:28.058266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:28.058266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:28.058266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:28.058266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:28.058266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:28.058266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:28.058266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:28.058266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:28.058266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:28.058654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.058654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:28.058933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.058933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:28.058933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:28.058933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:28.058933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:28.058933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:28.058933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.058933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:28.058933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:28.058933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.058933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:28.058933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:28.058933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.058933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:28.058933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:28.058933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.059460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:19:28.059460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:28.059460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:28.059460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:28.059460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:28.059460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:28.059460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:28.059460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:28.059460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:28.059460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:28.059460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:28.059460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:28.059460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:28.059460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:28.059460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:28.059460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:28.059460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:28.059460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:28.059460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:28.060172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:28.060172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.060172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:28.778165 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:28.778829 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:28.779089 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:28.780325 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:28.780748 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:28.780973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.781875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:28.781875 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:28.781875 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:28.781875 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:28.781875 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:28.781875 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:28.781875 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.781875 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:28.783369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.783369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:28.783369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:28.783369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:28.783369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.783369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:19:28.783369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:28.783369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:28.783369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:28.783369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:28.783369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:28.783369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:28.784094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:28.784094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:28.784094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:28.784094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:28.784094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:28.784094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:28.784094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:28.784094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:28.784094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:28.784094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:28.784094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:28.784094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:28.784094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:28.784766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.784766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:28.785270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.785270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:28.785270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:28.785270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:28.785270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:28.785270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:28.785270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.785270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:28.785270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:28.785270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.785270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:28.785270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:28.785270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.785270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:28.785270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:28.785270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.785838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:19:28.785838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:28.785838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:28.785838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:28.785838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:28.785838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:28.785838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:28.785838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:28.785838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:28.785838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:28.785838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:28.785838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:28.785838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:28.785838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:28.785838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:28.785838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:28.785838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:28.785838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:28.785838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:28.786334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:28.786334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.786334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:28.820428 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:28.820903 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:28.821083 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:28.821264 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:19:28.822036 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:28.822341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.823171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:28.823171 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:28.823171 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:28.823171 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:28.823171 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:28.823171 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:28.823171 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.823171 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:28.824365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.824365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:19:28.824365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:28.824365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:28.824365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:28.824365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:28.824365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:28.824365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:28.824365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:28.824365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:28.824365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:28.824365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:19:28.824365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:28.824365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:19:28.824365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:19:28.825200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:28.825200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:19:28.825200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:28.825200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:19:28.825200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:19:28.825200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:28.825200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:19:28.825200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.825200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:19:28.825650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.825650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:28.825650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:28.825650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:28.825650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.825650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:19:28.825650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:19:28.825650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:19:28.825650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:28.825650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:28.825650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:28.825650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:28.826232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:28.826232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:28.826232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:28.826232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:28.826232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:28.826232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:28.826232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:28.826232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:28.826232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:28.826232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:28.826232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:28.826232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:28.826232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:28.826232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:28.826879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:28.826879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:28.826879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.826879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:28.827272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.827272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:28.827272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:28.827272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:28.827272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:28.827272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:28.827272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.827272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:28.827272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:28.827272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.827272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:28.827272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:28.827272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.827272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:28.827272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:28.827272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.828085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:19:28.828085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:19:28.828085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:19:28.828085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:28.828085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:28.828085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:28.828085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:28.828085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:28.828085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:28.828085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:28.828085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:28.828085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:28.828085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:28.828085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:28.828085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:28.828085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:28.828085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:28.828085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:28.828919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:28.828919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:28.828919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:28.828919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:28.828919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:28.828919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.828919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:28.830148 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:28.830466 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:28.830629 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:28.830832 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:28.831181 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:28.831406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.831903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:28.831903 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:28.831903 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:28.831903 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:28.831903 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:28.831903 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:28.831903 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.831903 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:28.832696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.832696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:19:28.832696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:28.832696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:28.832696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:28.832696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:28.832696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:28.832696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:28.832696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:28.832696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:28.832696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:28.832696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:19:28.832696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:28.832696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:19:28.832696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:19:28.833152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:28.833152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:19:28.833152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:28.833152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:19:28.833152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:19:28.833152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:28.833152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:19:28.833152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.833152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:19:28.833569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.833569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:28.833569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:28.833569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:28.833569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.833569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:19:28.833569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:19:28.833569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:28.833569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:28.833569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:28.833569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:28.833959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:28.833959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:28.833959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:28.833959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:28.833959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:28.833959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:28.833959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:28.833959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:28.833959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:28.833959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:28.833959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:28.833959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:28.833959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:28.833959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:28.834353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:28.834353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:28.834353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.834353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:28.834706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.834706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:28.834706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:28.834706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:28.834706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:28.834706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:28.834706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.834706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:28.834706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:28.834706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.834706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:28.834706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:28.834706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.834706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:28.834706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:28.834706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.834706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:19:28.835219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:19:28.835219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:28.835219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:28.835219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:28.835219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:28.835219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:28.835219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:28.835219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:28.835219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:28.835219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:28.835219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:28.835219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:28.835219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:28.835219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:28.835219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:28.835219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:28.835219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:28.835219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:28.835219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:28.835726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:28.835726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:28.835726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.835726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:28.838983 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:28.839264 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:28.839371 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:28.839526 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:28.840324 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:28.840694 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:28.840951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.841596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:28.841596 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:28.841596 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:28.841596 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:28.841596 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:28.841596 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:28.841596 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.841596 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:28.842443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.842443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:28.842443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:28.842443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:28.842443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.842443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:19:28.842443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:28.842443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:28.842443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:28.842443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:28.842443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:28.842443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:28.842904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:28.842904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:28.842904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:28.842904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:28.842904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:28.842904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:28.842904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:28.842904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:28.842904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:28.842904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:28.842904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:28.842904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:28.842904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:28.843250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.843250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:28.843447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.843447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:28.843447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:28.843447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:28.843447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:28.843447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:28.843447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.843447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:28.843447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:28.843447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.843447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:28.843447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:28.843447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.843447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:28.843447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:28.843447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:28.843924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:19:28.843924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:28.843924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:28.843924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:28.843924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:28.843924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:28.843924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:28.843924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:28.843924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:28.843924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:28.843924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:28.843924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:28.843924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:28.843924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:28.843924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:28.843924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:28.843924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:28.843924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:28.843924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:28.844491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:28.844491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:28.844491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:33.020339 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:33.020706 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:33.020742 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:33.021373 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:33.081914 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:33.082217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.083277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:33.083277 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:33.083277 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:33.083277 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:33.083277 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:33.083277 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:33.083277 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.083277 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:33.084326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.063 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.084326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:33.084326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:33.084326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:33.084326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.084326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:19:33.084326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:33.084326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:33.084326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:33.084326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:33.084326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:33.084326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:33.084800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:33.084800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:33.084800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:33.084800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:33.084800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:33.084800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:33.084800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:33.084800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:33.084800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:33.084800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:33.084800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:33.084800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:33.084800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:33.085160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.085160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:33.085762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.085762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:33.085762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:33.085762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:33.085762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:33.085762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:33.085762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.085762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:33.085762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:33.085762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.085762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:33.085762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:33.085762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.085762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:33.085762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:33.085762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.086208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:19:33.086208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:33.086208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:33.086208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:33.086208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:33.086208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:33.086208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:33.086208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:33.086208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:33.086208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:33.086208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:33.086208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:33.086208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:33.086208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:33.086208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:33.086208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:33.086208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:33.086208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:33.086208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:33.086761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:33.086761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.086761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:33.779969 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:33.780544 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:33.780784 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:33.781955 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:33.782564 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:33.782986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.784258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:33.784258 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:33.784258 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:33.784258 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:33.784258 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:33.784258 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:33.784258 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.784258 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:33.786047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.786047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:33.786047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:33.786047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:33.786047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.786047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:19:33.786047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:33.786047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:33.786047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:33.786047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:33.786047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:33.786047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:33.786652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:33.786652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:33.786652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:33.786652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:33.786652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:33.786652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:33.786652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:33.786652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:33.786652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:33.786652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:33.786652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:33.786652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:33.786652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:33.787284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.787284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:33.788039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.788039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:33.788039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:33.788039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:33.788039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:33.788039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:33.788039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.788039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:33.788039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:33.788039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.788039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:33.788039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:33.788039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.788039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:33.788039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:33.788039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.788994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:19:33.788994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:33.788994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:33.788994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:33.788994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:33.788994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:33.788994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:33.788994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:33.788994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:33.788994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:33.788994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:33.788994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:33.788994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:33.788994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:33.788994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:33.788994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:33.788994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:33.788994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:33.788994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:33.790037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:33.790037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.790037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:33.820975 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:33.821368 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:33.821451 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:33.821596 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:19:33.822313 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:33.822693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.823575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:33.823575 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:33.823575 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:33.823575 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:33.823575 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:33.823575 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:33.823575 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.823575 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:33.824519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.824519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:19:33.824519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:33.824519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:33.824519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:33.824519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:33.824519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:33.824519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:33.824519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:33.824519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:33.824519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:33.824519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:19:33.824519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:33.824519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:19:33.824519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:19:33.825190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:33.825190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:19:33.825190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:33.825190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:19:33.825190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:19:33.825190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:33.825190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:19:33.825190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.825190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:19:33.825798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.825798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:33.825798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:33.825798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:33.825798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.825798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:19:33.825798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:19:33.825798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:19:33.825798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:33.825798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:33.825798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:33.825798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:33.826319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:33.826319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:33.826319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:33.826319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:33.826319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:33.826319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:33.826319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:33.826319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:33.826319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:33.826319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:33.826319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:33.826319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:33.826319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:33.826319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:33.826920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:33.826920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:33.826920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.826920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:33.826920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.826920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:33.826920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:33.826920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:33.826920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:33.826920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:33.826920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.826920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:33.826920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:33.826920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.827598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:33.827598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:33.827598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.827598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:33.827598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:33.827598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.827598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:19:33.827598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:19:33.827598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:19:33.827598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:33.827598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:33.827598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:33.827598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:33.827598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:33.827598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:33.827598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:33.827598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:33.827598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:33.827598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:33.828462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:33.828462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:33.828462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:33.828462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:33.828462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:33.828462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:33.828462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:33.828462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:33.828462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:33.828462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:33.828462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.828462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:33.833890 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:33.834134 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:33.834244 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:33.834372 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:33.834734 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:33.835004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.835608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:33.835608 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:33.835608 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:33.835608 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:33.835608 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:33.835608 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:33.835608 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.835608 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:33.836508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.836508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:19:33.836508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:33.836508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:33.836508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:33.836508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:33.836508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:33.836508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:33.836508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:33.836508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:33.836508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:33.836508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:19:33.836508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:33.836508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:19:33.836508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:19:33.836977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:33.836977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:19:33.836977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:33.836977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:19:33.836977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:19:33.836977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:33.836977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:19:33.836977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.836977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:19:33.838156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.838156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:33.838156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:33.838156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:33.838156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.838156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:19:33.838156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:19:33.838156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:33.838156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:33.838156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:33.838156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:33.838518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:33.838518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:33.838518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:33.838518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:33.838518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:33.838518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:33.838518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:33.838518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:33.838518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:33.838518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:33.838518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:33.838518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:33.838518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:33.838518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:33.839240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:33.839240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:33.839240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.839240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:33.840270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.840270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:33.840270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:33.840270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:33.840270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:33.840270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:33.840270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.840270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:33.840270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:33.840270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.840270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:33.840270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:33.840270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.840270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:33.840270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:33.840270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.840270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:19:33.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:19:33.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:33.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:33.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:33.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:33.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:33.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:33.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:33.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:33.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:33.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:33.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:33.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:33.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:33.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:33.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:33.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:33.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:33.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:33.841345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:33.841345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:33.841345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.841345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:33.841345 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:33.841345 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:33.841345 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:33.841745 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:33.842302 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:33.842631 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:33.842907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.843833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:33.843833 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:33.843833 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:33.843833 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:33.843833 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:33.843833 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:33.843833 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.843833 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:33.844776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.844776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:33.844776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:33.844776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:33.844776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.844776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:19:33.844776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:33.844776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:33.844776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:33.844776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:33.844776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:33.844776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:33.845175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:33.845175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:33.845175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:33.845175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:33.845175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:33.845175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:33.845175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:33.845175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:33.845175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:33.845175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:33.845175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:33.845175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:33.845175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:33.845544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.845544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:33.846100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.846100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:33.846100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:33.846100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:33.846100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:33.846100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:33.846100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.846100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:33.846100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:33.846100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.846100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:33.846100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:33.846100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.846100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:33.846100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:33.846100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:33.846592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:19:33.846592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:33.846592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:33.846592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:33.846592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:33.846592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:33.846592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:33.846592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:33.846592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:33.846592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:33.846592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:33.846592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:33.846592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:33.846592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:33.846592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:33.846592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:33.846592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:33.846592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:33.846592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:33.847221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:33.847221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:33.847221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:38.020932 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:38.021896 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:38.021896 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:38.022203 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:38.059180 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:38.059363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.059891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:38.059891 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:38.059891 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:38.059891 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:38.059891 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:38.059891 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:38.059891 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.059891 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:38.060639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.060639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:38.060639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:38.060639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:38.060639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.060639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:19:38.060639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:38.060639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:38.060639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:38.060639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:38.060639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:38.060639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:38.061122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:38.061122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:38.061122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:38.061122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:38.061122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:38.061122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:38.061122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:38.061122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:38.061122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:38.061122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:38.061122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:38.061122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:38.061122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:38.061509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.061509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:38.061916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.061916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:38.061916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:38.061916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:38.061916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:38.061916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:38.061916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.061916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:38.061916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:38.061916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.061916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:38.061916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:38.061916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.061916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:38.061916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:38.061916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.062392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:19:38.062392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:38.062392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:38.062392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:38.062392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:38.062392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:38.062392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:38.062392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:38.062392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:38.062392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:38.062392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:38.062392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:38.062392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:38.062392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:38.062392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:38.062392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:38.062392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:38.062392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:38.062392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:38.063496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:38.063496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.063496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:38.780716 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:38.781326 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:38.781550 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:38.782586 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:38.782937 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:38.783183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.784977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:38.784977 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:38.784977 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:38.784977 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:38.784977 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:38.784977 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:38.784977 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.784977 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:38.786597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.786597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:38.786597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:38.786597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:38.786597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.786597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:19:38.786597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:38.786597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:38.786597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:38.786597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:38.786597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:38.786597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:38.787255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:38.787255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:38.787255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:38.787255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:38.787255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:38.787255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:38.787255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:38.787255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:38.787255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:38.787255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:38.787255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:38.787255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:38.787255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:38.787922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.787922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:38.788393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.788393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:38.788393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:38.788393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:38.788393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:38.788393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:38.788393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.788393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:38.788393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:38.788393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.788393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:38.788393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:38.788393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.788393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:38.788393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:38.788393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.789117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:19:38.789117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:38.789117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:38.789117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:38.789117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:38.789117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:38.789117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:38.789117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:38.789117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:38.789117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:38.789117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:38.789117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:38.789117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:38.789117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:38.789117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:38.789117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:38.789117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:38.789117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:38.789117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:38.790499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:38.790499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.790499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:38.821559 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:38.821973 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:38.822158 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:38.822346 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:19:38.822827 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:38.823122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.823899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:38.823899 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:38.823899 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:38.823899 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:38.823899 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:38.823899 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:38.823899 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.823899 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:38.825071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.825071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:19:38.825071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:38.825071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:38.825071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:38.825071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:38.825071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:38.825071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:38.825071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:38.825071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:38.825071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:38.825071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:19:38.825071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:38.825071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:19:38.825071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:19:38.825860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:38.825860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:19:38.825860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:38.825860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:19:38.825860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:19:38.825860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:38.825860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:19:38.825860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.825860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:19:38.826359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.826359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:38.826359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:38.826359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:38.826359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.826359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:19:38.826359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:19:38.826359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:19:38.826359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:38.826359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:38.826359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:38.826359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:38.827098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:38.827098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:38.827098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:38.827098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:38.827098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:38.827098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:38.827098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:38.827098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:38.827098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:38.827098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:38.827098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:38.827098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:38.827098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:38.827098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:38.827807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:38.827807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:38.827807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.827807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:38.827807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.827807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:38.827807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:38.827807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:38.827807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:38.827807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:38.827807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.827807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:38.827807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:38.827807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.828515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:38.828515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:38.828515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.828515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:38.828515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:38.828515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.828515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:19:38.828515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:19:38.828515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:19:38.828515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:38.828515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:38.828515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:38.828515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:38.828515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:38.828515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:38.828515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:38.828515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:38.828515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:38.828515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:38.829355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:38.829355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:38.829355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:38.829355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:38.829355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:38.829355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:38.829355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:38.829355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:38.829355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:38.829355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:38.829355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.829355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:38.834209 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:38.834542 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:38.834663 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:38.834857 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:38.835301 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:38.836032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.836591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:38.836591 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:38.836591 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:38.836591 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:38.836591 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:38.836591 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:38.836591 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.836591 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:38.843659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.843659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:19:38.843659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:38.843659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:38.843659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:38.843659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:38.843659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:38.843659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:38.843659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:38.843659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:38.843659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:38.843659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:19:38.843659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:38.843659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:19:38.843659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:19:38.844749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:38.844749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:19:38.844749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:38.844749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:19:38.844749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:19:38.844749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:38.844749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:19:38.844749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.844749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:19:38.844749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.844749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:38.844749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:38.844749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:38.844749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.845472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:19:38.845472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:19:38.845472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:38.845472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:38.845472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:38.845472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:38.845472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:38.845472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:38.845472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:38.845472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:38.845472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:38.845472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:38.845472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:38.846362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:38.846362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:38.846362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:38.846362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:38.846362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:38.846362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:38.846362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:38.846362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:38.846362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:38.846362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.846362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:38.846362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.846990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:38.846990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:38.846990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:38.846990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:38.846990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:38.846990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.846990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:38.846990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:38.846990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.846990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:38.846990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:38.846990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.846990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:38.846990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:38.846990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.846990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:19:38.846990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:19:38.846990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:38.846990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:38.848041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:38.848041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:38.848041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:38.848041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:38.848041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:38.848041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:38.848041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:38.848041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:38.848041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:38.848041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:38.848041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:38.848041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:38.848041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:38.848041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:38.848041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:38.848041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:38.848041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:38.848041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:38.848041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.848041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:38.849388 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:38.849388 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:38.849388 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:38.849388 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:38.849388 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:38.849388 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:38.849857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.849857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:38.849857 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:38.849857 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:38.849857 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:38.849857 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:38.849857 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:38.849857 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.849857 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:38.849857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.849857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:38.849857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:38.849857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:38.849857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.849857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:19:38.850499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:38.850499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:38.850499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:38.850499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:38.850499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:38.850499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:38.850499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:38.850499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:38.850499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:38.850499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:38.850499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:38.850499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:38.850499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:38.850499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:38.850838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:38.850838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:38.850838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:38.850838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:38.850838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:38.850838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.850838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:38.850838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.850838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:38.850838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:38.850838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:38.850838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:38.850838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:38.851197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.851197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:38.851197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:38.851197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.851197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:38.851197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:38.851197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.851197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:38.851197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:38.851197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:38.851197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:19:38.851197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:38.851197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:38.851197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:38.851197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:38.851197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:38.851197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:38.851197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:38.851197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:38.851197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:38.851542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:38.851542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:38.851542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:38.851542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:38.851542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:38.851542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:38.851542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:38.851542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:38.851542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:38.851542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:38.851542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:38.851542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:43.021557 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:43.022215 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:43.022215 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:43.022891 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:43.055789 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:43.055789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.056883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:43.056883 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:43.056883 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:43.056883 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:43.056883 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:43.056883 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:43.056883 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.056883 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:43.057666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.057666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:43.057666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:43.057666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:43.057666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.057666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:19:43.057666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:43.057666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:43.057666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:43.057666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:43.057666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:43.057666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:43.058065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:43.058065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:43.058065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:43.058065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:43.058065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:43.058065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:43.058065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:43.058065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:43.058065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:43.058065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:43.058065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:43.058065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:43.058065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:43.058444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.058444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:43.058444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.058444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:43.058444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:43.058444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:43.058444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:43.058444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:43.058444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.058444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:43.058444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:43.058444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.058444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:43.058444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:43.058444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.058929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:43.058929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:43.058929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.058929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:19:43.058929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:43.058929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:43.058929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:43.058929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:43.058929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:43.058929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:43.058929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:43.058929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:43.058929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:43.058929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:43.058929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:43.058929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:43.058929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:43.058929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:43.058929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:43.059464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:43.059464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:43.059464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:43.059464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:43.059464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.059464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:43.781801 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:43.782038 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:43.782192 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:43.783289 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:43.784757 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:43.784757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.785089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:43.785089 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:43.785089 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:43.785089 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:43.785089 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:43.785089 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:43.785089 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.785089 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:43.786594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.786594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:43.786594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:43.786594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:43.786594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.786594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:19:43.786594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:43.786594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:43.786594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:43.786594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:43.786594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:43.786594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:43.787186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:43.787186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:43.787186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:43.787186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:43.787186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:43.787186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:43.787186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:43.787186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:43.787186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:43.787186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:43.787186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:43.787186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:43.787186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:43.788485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.788485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:43.788485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.788485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:43.788485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:43.788485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:43.788485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:43.788485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:43.788485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.788485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:43.788485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:43.788485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.788485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:43.788485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:43.788485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.789328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:43.789328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:43.789328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.789328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:19:43.789328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:43.789328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:43.789328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:43.789328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:43.789328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:43.789328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:43.789328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:43.789328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:43.789328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:43.789328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:43.789328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:43.789328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:43.789328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:43.789328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:43.789328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:43.790166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:43.790166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:43.790166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:43.790166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:43.790166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.790166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:43.821981 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:43.822350 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:43.822523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:43.822730 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:19:43.824250 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:43.824250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.824506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:43.824506 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:43.824506 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:43.824506 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:43.824506 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:43.824506 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:43.824506 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.824506 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:43.825722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.825722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:19:43.825722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:43.825722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:43.825722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:43.825722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:43.825722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:43.825722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:43.825722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:43.825722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:43.825722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:43.825722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:19:43.825722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:43.825722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:19:43.825722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:19:43.826541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:43.826541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:19:43.826541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:43.826541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:19:43.826541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:19:43.826541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:43.826541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:19:43.826541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.826541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:19:43.827057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.827057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:43.827057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:43.827057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:43.827057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.827057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:19:43.827057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:19:43.827057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:19:43.827057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:43.827057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:43.827057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:43.827057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:43.827419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:43.827419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:43.827419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:43.827419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:43.827419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:43.827419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:43.827419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:43.827419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:43.827419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:43.827419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:43.827419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:43.827419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:43.827419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:43.827419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:43.828046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:43.828046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:43.828046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.828046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:43.829647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.829647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:43.829647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:43.829647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:43.829647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:43.829647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:43.829647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.829647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:43.829647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:43.829647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.829647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:43.829647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:43.829647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.829647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:43.829647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:43.829647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.830165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:19:43.830165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:19:43.830165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:19:43.830165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:43.830165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:43.830165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:43.830165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:43.830165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:43.830165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:43.830165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:43.830165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:43.830165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:43.830165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:43.830165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:43.830165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:43.830165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:43.830165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:43.830165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:43.830651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:43.830651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:43.830651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:43.830651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:43.830651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:43.830651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.830651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:43.834665 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:43.834945 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:43.835057 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:43.835177 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:43.835620 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:43.836046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.837106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:43.837106 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:43.837106 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:43.837106 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:43.837106 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:43.837106 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:43.837106 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.837106 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:43.837705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.837705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:19:43.837705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:43.837705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:43.837705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:43.837705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:43.837705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:43.837705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:43.837705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:43.837705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:43.837705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:43.837705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:19:43.837705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:43.837705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:19:43.837705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:19:43.838177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:43.838177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:19:43.838177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:43.838177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:19:43.838177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:19:43.838177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:43.838177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:19:43.838177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.838177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:19:43.838554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.838554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:43.838554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:43.838554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:43.838554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.838554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:19:43.838554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:19:43.838554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:43.838554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:43.838554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:43.838554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:43.838986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:43.838986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:43.838986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:43.838986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:43.838986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:43.838986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:43.838986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:43.838986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:43.838986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:43.838986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:43.838986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:43.838986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:43.838986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:43.838986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:43.839406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:43.839406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:43.839406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.839406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:43.839589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.839589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:43.839589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:43.839589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:43.839589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:43.839589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:43.839589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.839589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:43.839589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:43.839589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.839589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:43.839589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:43.839589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.839589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:43.839589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:43.839589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.839589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:19:43.840157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:19:43.840157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:43.840157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:43.840157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:43.840157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:43.840157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:43.840157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:43.840157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:43.840157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:43.840157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:43.840157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:43.840157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:43.840157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:43.840157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:43.840157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:43.840157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:43.840157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:43.840157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:43.840157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:43.840721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:43.840721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:43.840721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.840721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:43.841617 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:43.841884 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:43.841985 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:43.842140 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:43.842770 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:43.842974 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:43.842974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.843503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:43.843503 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:43.843503 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:43.843503 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:43.843503 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:43.843503 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:43.843503 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.843503 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:43.844432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.844432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:43.844432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:43.844432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:43.844432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.844432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:19:43.844432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:43.844432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:43.844432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:43.844432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:43.844432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:43.844432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:43.845122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:43.845122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:43.845122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:43.845122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:43.845122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:43.845122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:43.845122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:43.845122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:43.845122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:43.845122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:43.845122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:43.845122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:43.845122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:43.845752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.845752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:43.845752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.845752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:43.845752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:43.845752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:43.845752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:43.845752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:43.845752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.845752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:43.845752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:43.845752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.845752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:43.845752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:43.845752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.846500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:43.846500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:43.846500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:43.846500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:19:43.846500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:43.846500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:43.846500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:43.846500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:43.846500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:43.846500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:43.846500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:43.846500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:43.846500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:43.846500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:43.846500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:43.846500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:43.846500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:43.846500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:43.846500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:43.847435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:43.847435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:43.847435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:43.847435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:43.847435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:43.847435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:48.022257 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:48.025335 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:48.025335 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:48.025335 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:48.062560 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:48.062983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.063883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:48.063883 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:48.063883 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:48.063883 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:48.063883 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:48.063883 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:48.063883 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.063883 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:48.066540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.066540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:48.066540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:48.066540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:48.066540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.066540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:19:48.066540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:48.066540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:48.066540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:48.066540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:48.066540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:48.066540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:48.066969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:48.066969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:48.066969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:48.066969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:48.066969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:48.066969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:48.066969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:48.066969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:48.066969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:48.066969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:48.066969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:48.066969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:48.066969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:48.067387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.067387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:48.067387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.067387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:48.067387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:48.067387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:48.067387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:48.067387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:48.067387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.067387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:48.067387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:48.067387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.067387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:48.067387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:48.067387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.067818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:48.067818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:48.067818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.067818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:19:48.067818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:48.067818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:48.067818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:48.067818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:48.067818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:48.067818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:48.067818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:48.067818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:48.067818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:48.067818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:48.067818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:48.067818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:48.067818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:48.067818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:48.067818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:48.068467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:48.068467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:48.068467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:48.068467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:48.068467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.068467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:48.782327 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:48.784019 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:48.784019 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:48.784702 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:48.785082 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:48.785551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.789346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:48.789346 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:48.789346 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:48.789346 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:48.789346 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:48.789346 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:48.789346 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.789346 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:48.789346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.789346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:48.789346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:48.789346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:48.789346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.789346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:19:48.789346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:48.790159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:48.790159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:48.790159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:48.790159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:48.790159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:48.790159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:48.790159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:48.790159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:48.790159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:48.790159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:48.790159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:48.790159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:48.790159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:48.790159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:48.790952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:48.790952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:48.790952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:48.790952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:48.790952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.790952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:48.794438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.794438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:48.794438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:48.794438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:48.794438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:48.794438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:48.794438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.794438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:48.794438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:48.794438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.794438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:48.794438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:48.794438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.794438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:48.794438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:48.794438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.795274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:19:48.795274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:48.795274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:48.795274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:48.795274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:48.795274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:48.795274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:48.795274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:48.795274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:48.795274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:48.795274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:48.795274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:48.795274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:48.795274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:48.795274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:48.795274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:48.795274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:48.795274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:48.795274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:48.796249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:48.796249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.796249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:48.822427 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:48.824100 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:48.824100 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:48.824100 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:19:48.824343 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:48.824725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.828834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:48.828834 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:48.828834 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:48.828834 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:48.828834 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:48.828834 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:48.828834 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.828834 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:48.828834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.828834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:19:48.828834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:48.828834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:48.828834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:48.828834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:48.828834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:48.828834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:48.828834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:48.828834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:48.829380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:48.829380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:19:48.829380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:48.829380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:19:48.829380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:19:48.829380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:48.829380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:19:48.829380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:48.829380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:19:48.829380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:19:48.829380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:48.829380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:19:48.829380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.829380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:19:48.829380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.829803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:48.829803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:48.829803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:48.829803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.829803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:19:48.829803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:19:48.829803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:19:48.829803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:48.829803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:48.829803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:48.829803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:48.829803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:48.829803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:48.830118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:48.830118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:48.830118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:48.830118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:48.830118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:48.830118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:48.830118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:48.830118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:48.830118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:48.830118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:48.830118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:48.830118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:48.830118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:48.830118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:48.830438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.830438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:48.831272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.831272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:48.831272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:48.831272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:48.831272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:48.831272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:48.831272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.831272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:48.831272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:48.831272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.831272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:48.831272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:48.831272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.831272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:48.831272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:48.831272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.831733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:19:48.831733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:19:48.831733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:19:48.831733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:48.831733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:48.831733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:48.831733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:48.831733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:48.831733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:48.831733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:48.831733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:48.831733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:48.831733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:48.831733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:48.831733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:48.831733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:48.831733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:48.831733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:48.832306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:48.832306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:48.832306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:48.832306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:48.832306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:48.832306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.832306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:48.835006 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:48.835472 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:48.835472 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:48.835472 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:48.835866 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:48.836209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.839868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:48.839868 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:48.839868 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:48.839868 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:48.839868 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:48.839868 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:48.839868 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.839868 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:48.839868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.839868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:19:48.839868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:48.839868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:48.839868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:48.839868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:48.839868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:48.839868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:48.839868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:48.839868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:48.840875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:48.840875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:19:48.840875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:48.840875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:19:48.840875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:19:48.840875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:48.840875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:19:48.840875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:48.840875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:19:48.840875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:19:48.840875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:48.840875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:19:48.840875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.840875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:19:48.840875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.841566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:48.841566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:48.841566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:48.841566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.841566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:19:48.841566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:19:48.841566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:48.841566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:48.841566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:48.841566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:48.841566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:48.841566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:48.841566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:48.842053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:48.842053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:48.842053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:48.842053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:48.842053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:48.842053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:48.842053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:48.842053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:48.842053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:48.842053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:48.842053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:48.842053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:48.842053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:48.842409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.842409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:48.843045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.843045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:48.843045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:48.843045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:48.843045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:48.843045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:48.843045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.843045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:48.843045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:48.843045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.843045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:48.843045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:48.843045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.843045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:48.843045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:48.843045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.843045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:19:48.843638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:19:48.843638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:48.843638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:48.843638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:48.843638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:48.843638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:48.843638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:48.843638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:48.843638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:48.843638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:48.843638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:48.843638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:48.843638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:48.843638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:48.843638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:48.843638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:48.843638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:48.843638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:48.843638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:48.844188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:48.844188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:48.844188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.844188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:48.844866 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:48.845594 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:48.845594 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:48.845594 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:48.846030 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:48.846397 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:48.846624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.848630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:48.848630 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:48.848630 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:48.848630 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:48.848630 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:48.848630 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:48.848630 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.848630 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:48.848630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.848630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:48.848630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:48.848630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:48.848630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.848630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:19:48.848630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:48.849306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:48.849306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:48.849306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:48.849306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:48.849306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:48.849306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:48.849306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:48.849306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:48.849306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:48.849306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:48.849306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:48.849306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:48.849306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:48.849306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:48.849656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:48.849656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:48.849656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:48.849656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:48.849656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.849656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:48.849914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.849914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:48.849914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:48.849914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:48.849914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:48.849914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:48.849914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.849914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:48.849914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:48.849914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.849914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:48.849914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:48.849914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.849914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:48.849914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:48.849914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:48.850425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:19:48.850425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:48.850425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:48.850425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:48.850425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:48.850425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:48.850425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:48.850425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:48.850425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:48.850425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:48.850425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:48.850425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:48.850425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:48.850425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:48.850425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:48.850425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:48.850425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:48.850425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:48.850425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:48.851036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:48.851036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:48.851036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:53.023605 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:53.024564 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:53.024791 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:53.026044 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:53.059275 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:53.059797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.060644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:53.060644 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:53.060644 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:53.060644 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:53.060644 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:53.060644 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:53.060644 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.060644 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:53.061752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.061752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:53.061752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:53.061752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:53.061752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.061752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:19:53.061752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:53.061752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:53.061752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:53.061752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:53.061752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:53.061752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:53.062386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:53.062386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:53.062386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:53.062386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:53.062386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:53.062386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:53.062386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:53.062386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:53.062386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:53.062386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:53.062386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:53.062386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:53.062386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:53.062992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.062992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:53.062992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.062992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:53.062992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:53.062992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:53.062992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:53.062992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:53.062992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.062992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:53.062992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:53.062992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.062992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:53.062992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:53.062992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.063721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:53.063721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:53.063721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.063721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:19:53.063721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:53.063721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:53.063721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:53.063721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:53.063721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:53.063721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:53.063721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:53.063721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:53.063721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:53.063721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:53.063721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:53.063721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:53.063721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:53.063721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:53.063721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:53.064585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:53.064585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:53.064585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:53.064585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:53.064585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.064585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:53.783472 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:53.784291 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:53.784427 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:53.785726 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:53.786090 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:53.786309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.787280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:53.787280 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:53.787280 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:53.787280 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:53.787280 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:53.787280 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:53.787280 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.787280 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:53.788875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.788875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:53.788875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:53.788875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:53.788875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.788875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:19:53.788875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:53.788875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:53.788875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:53.788875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:53.788875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:53.788875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:53.789499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:53.789499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:53.789499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:53.789499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:53.789499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:53.789499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:53.789499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:53.789499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:53.789499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:53.789499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:53.789499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:53.789499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:53.789499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:53.790043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.790043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:53.791886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.791886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:53.791886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:53.791886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:53.791886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:53.791886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:53.791886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.791886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:53.791886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:53.791886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.791886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:53.791886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:53.791886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.791886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:53.791886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:53.791886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.793470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:19:53.793470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:53.793470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:53.793470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:53.793470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:53.793470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:53.793470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:53.793470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:53.793470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:53.793470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:53.793470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:53.793470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:53.793470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:53.793470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:53.793470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:53.793470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:53.793470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:53.793470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:53.793470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:53.794892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:53.794892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.794892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:53.823191 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:53.823695 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:53.823892 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:53.824068 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:19:53.824564 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:53.824825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.825452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:53.825452 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:53.825452 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:53.825452 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:53.825452 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:53.825452 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:53.825452 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.825452 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:53.826650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.826650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:19:53.826650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:53.826650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:53.826650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:53.826650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:53.826650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:53.826650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:53.826650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:53.826650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:53.826650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:53.826650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:19:53.826650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:53.826650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:19:53.826650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:19:53.827408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:53.827408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:19:53.827408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:53.827408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:19:53.827408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:19:53.827408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:53.827408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:19:53.827408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.827408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:19:53.828014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.828014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:53.828014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:53.828014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:53.828014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.828014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:19:53.828014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:19:53.828014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:19:53.828014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:53.828014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:53.828014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:53.828014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:53.828998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:53.828998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:53.828998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:53.828998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:53.828998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:53.828998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:53.828998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:53.828998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:53.828998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:53.828998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:53.828998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:53.828998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:53.828998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:53.828998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:53.830662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:53.830662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:53.830662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.830662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:53.830662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.830662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:53.830662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:53.830662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:53.830662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:53.830662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:53.830662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.830662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:53.830662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:53.830662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.831581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:53.831581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:53.831581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.831581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:53.831581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:53.831581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.831581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:19:53.831581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:19:53.831581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:19:53.831581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:53.831581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:53.831581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:53.831581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:53.831581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:53.831581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:53.831581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:53.831581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:53.831581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:53.831581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:53.832633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:53.832633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:53.832633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:53.832633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:53.832633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:53.832633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:53.832633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:53.832633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:53.832633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:53.832633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:53.832633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.832633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:53.835540 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:53.835900 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:53.836010 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:53.836151 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:53.836668 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:53.837103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.837821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:53.837821 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:53.837821 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:53.837821 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:53.837821 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:53.837821 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:53.837821 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.837821 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:53.838593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.838593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:19:53.838593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:53.838593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:53.838593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:53.838593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:53.838593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:53.838593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:53.838593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:53.838593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:53.838593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:53.838593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:19:53.838593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:53.838593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:19:53.838593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:19:53.839314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:53.839314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:19:53.839314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:53.839314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:19:53.839314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:19:53.839314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:53.839314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:19:53.839314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.839314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:19:53.839771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.839771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:53.839771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:53.839771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:53.839771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.839771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:19:53.839771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:19:53.839771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:53.839771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:53.839771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:53.839771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:53.840300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:53.840300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:53.840300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:53.840300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:53.840300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:53.840300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:53.840300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:53.840300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:53.840300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:53.840300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:53.840300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:53.840300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:53.840300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:53.840300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:53.841059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:53.841059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:53.841059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.841059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:53.841059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.841059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:53.841059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:53.841059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:53.841059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:53.841059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:53.841059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.841059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:53.841059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:53.841059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.842048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:53.842048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:53.842048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.842048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:53.842048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:53.842048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.842048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:19:53.842048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:19:53.842048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:53.842048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:53.842048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:53.842048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:53.842048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:53.842048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:53.842048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:53.842048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:53.842048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:53.842048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:53.842048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:53.844557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:53.844557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:53.844557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:53.844557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:53.844557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:53.844557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:53.844557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:53.844557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:53.844557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:53.844557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.844557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:53.845815 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:53.845815 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:53.846037 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:53.846037 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:53.846470 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:53.846804 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:53.847050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.847834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:53.847834 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:53.847834 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:53.847834 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:53.847834 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:53.847834 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:53.847834 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.847834 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:53.848623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.848623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:53.848623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:53.848623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:53.848623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.848623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:19:53.848623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:53.848623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:53.848623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:53.848623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:53.848623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:53.848623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:53.848927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:53.848927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:53.848927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:53.848927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:53.848927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:53.848927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:53.848927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:53.848927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:53.848927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:53.848927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:53.848927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:53.848927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:53.848927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:53.849175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.849175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:53.849628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.849628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:53.849628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:53.849628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:53.849628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:53.849628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:53.849628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.849628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:53.849628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:53.849628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.849628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:53.849628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:53.849628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.849628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:53.849628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:53.849628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:53.849998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:19:53.849998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:53.849998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:53.849998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:53.849998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:53.849998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:53.849998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:53.849998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:53.849998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:53.849998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:53.849998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:53.849998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:53.849998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:53.849998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:53.849998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:53.849998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:53.849998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:53.849998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:53.849998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:53.850430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:53.850430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:53.850430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:58.024504 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:58.025607 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:58.025607 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:58.025929 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:58.069149 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:58.069507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.072182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:58.072182 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:58.072182 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:58.072182 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:58.072182 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:58.072182 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:58.072182 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.072182 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:58.072182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.072182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:58.072182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:58.072182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:58.072182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.072182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:19:58.072182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:58.072621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:58.072621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:58.072621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:58.072621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:58.072621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:58.072621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:58.072621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:58.072621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:58.072621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:58.072621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:58.072621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:58.072621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:58.072621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:58.072621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:58.072970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:58.072970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:58.072970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:58.072970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:58.072970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.072970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:58.072970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.072970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:58.072970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:58.072970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:58.072970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:58.072970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:58.072970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.072970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:58.073339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:58.073339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.073339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:58.073339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:58.073339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.073339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:58.073339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:58.073339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.073339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:19:58.073339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:58.073339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:58.073339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:58.073339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:58.073339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:58.073339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:58.073339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:58.073339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:58.073339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:58.073339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:58.073339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:58.073339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:58.073940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:58.073940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:58.073940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:58.073940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:58.073940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:58.073940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:58.073940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:58.073940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.073940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:58.784026 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:58.785231 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:58.785231 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:58.786050 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:58.786700 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:58.787050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.790713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:58.790713 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:58.790713 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:58.790713 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:58.790713 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:58.790713 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:58.790713 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.790713 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:58.790713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.790713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:58.790713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:58.790713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:58.790713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.790713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:19:58.790713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:58.791337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:58.791337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:58.791337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:58.791337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:58.791337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:58.791337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:58.791337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:58.791337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:58.791337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:58.791337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:58.791337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:58.791337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:58.791337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:58.791337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:58.791865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:58.791865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:58.791865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:58.791865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:58.791865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.791865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:58.791865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.791865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:58.791865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:58.791865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:58.791865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:58.791865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:58.791865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.791865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:58.792376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:58.792376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.792376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:58.792376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:58.792376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.792376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:58.792376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:58.792376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.792376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:19:58.792376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:58.792376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:58.792376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:58.792376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:58.792376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:58.792376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:58.792376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:58.792376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:58.792376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:58.792376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:58.792376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:58.793154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:58.793154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:58.793154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:58.793154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:58.793154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:58.793154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:58.793154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:58.793154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:58.793154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.793154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:58.823883 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:58.825122 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:58.825122 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:58.825122 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:19:58.825840 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:58.826260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.830722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:58.830722 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:58.830722 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:58.830722 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:58.830722 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:58.830722 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:58.830722 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.830722 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:58.830722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.830722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:19:58.830722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:58.830722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:58.830722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:58.830722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:58.830722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:58.830722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:58.830722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:58.830722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:58.831891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:58.831891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:19:58.831891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:58.831891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:19:58.831891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:19:58.831891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:58.831891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:19:58.831891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:58.831891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:19:58.831891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:19:58.831891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:58.831891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:19:58.831891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.831891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:19:58.834204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.834204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:58.834204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:58.834204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:58.834204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.834204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:19:58.834204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:19:58.834204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:19:58.834204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:58.834204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:58.834204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:58.834204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:58.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:58.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:58.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:58.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:58.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:58.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:58.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:58.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:58.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:58.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:58.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:58.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:58.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:58.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:58.835308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:58.835308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:58.835308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.835308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:58.836651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.836651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:58.836651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:58.836651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:58.836651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:58.836651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:58.836651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.836651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:58.836651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:58.836651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.836651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:58.836651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:58.836651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.836651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:58.836651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:58.836651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.837083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:19:58.837083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:19:58.837083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:19:58.837083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:58.837083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:58.837083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:58.837083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:58.837083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:58.837083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:58.837083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:58.837083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:58.837083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:58.837083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:58.837083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:58.837083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:58.837083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:58.837083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:58.837083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:58.837469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:58.837469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:58.837469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:58.837469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:58.837469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:58.837469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.837469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:58.837469 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:58.838375 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:58.838375 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:58.838618 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:58.839168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:58.839651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.840735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:58.840735 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:58.840735 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:58.840735 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:58.840735 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:58.840735 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:58.840735 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.840735 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:58.841742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.841742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:19:58.841742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:58.841742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:58.841742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:58.841742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:58.841742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:58.841742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:58.841742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:58.841742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:19:58.841742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:58.841742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:19:58.841742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:58.841742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:19:58.841742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:19:58.842391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:58.842391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:19:58.842391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:58.842391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:19:58.842391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:19:58.842391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:58.842391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:19:58.842391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.842391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:19:58.843624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.843624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:58.843624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:58.843624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:58.843624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.843624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:19:58.843624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:19:58.843624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:58.843624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:58.843624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:58.843624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:58.843996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:58.843996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:58.843996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:58.843996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:58.843996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:58.843996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:58.843996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:58.843996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:58.843996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:58.843996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:58.843996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:58.843996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:58.843996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:58.843996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:58.844291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:58.844291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:58.844291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.844291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:58.845338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.845338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:58.845338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:58.845338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:58.845338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:58.845338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:58.845338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.845338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:58.845338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:58.845338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.845338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:58.845338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:58.845338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.845338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:58.845338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:58.845338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.845338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:19:58.846046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:19:58.846046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:19:58.846046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:19:58.846046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:58.846046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:58.846046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:58.846046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:58.846046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:58.846046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:58.846046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:58.846046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:58.846046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:58.846046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:58.846046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:58.846046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:58.846046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:58.846046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:58.846046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:58.846046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:58.846584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:58.846584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:58.846584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.846584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:19:58.846584 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:19:58.847293 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:19:58.847293 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:19:58.847293 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:19:58.847667 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:58.848008 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:19:58.848273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.849005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:19:58.849005 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:58.849005 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:19:58.849005 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:19:58.849005 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:58.849005 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:19:58.849005 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.849005 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:19:58.849956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.849956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:19:58.849956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:58.849956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:19:58.849956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.849956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:19:58.849956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:58.849956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:58.849956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:58.849956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:58.849956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:58.849956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:58.850301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:19:58.850301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:58.850301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:19:58.850301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:58.850301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:19:58.850301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:19:58.850301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:58.850301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:19:58.850301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:58.850301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:19:58.850301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:19:58.850301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:58.850301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:19:58.850558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.850558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:19:58.851705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.851705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:19:58.851705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:19:58.851705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:19:58.851705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:19:58.851705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:19:58.851705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.851705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:19:58.851705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:19:58.851705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.851705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:19:58.851705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:19:58.851705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.851705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:19:58.851705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:19:58.851705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:19:58.852124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:19:58.852124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:19:58.852124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:58.852124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:58.852124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:19:58.852124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:19:58.852124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:19:58.852124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:19:58.852124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:19:58.852124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:19:58.852124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:19:58.852124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:19:58.852124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:19:58.852124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:19:58.852124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:19:58.852124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:19:58.852124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:19:58.852124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:19:58.852124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:19:58.852560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:19:58.852560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:19:58.852560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:03.025347 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:03.027998 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:03.027998 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:03.027998 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:03.041209 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=243 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:20:03.061325 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:03.061632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.064394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:03.064394 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:03.064394 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:03.064394 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:03.064394 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:03.064394 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:03.064394 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.064394 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:03.064394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.064394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:03.064394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:03.064394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:03.064394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.064394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:20:03.064394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:03.064997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:03.064997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:03.064997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:03.064997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:03.064997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:03.064997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:03.064997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:03.064997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:03.064997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:03.064997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:03.064997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:03.064997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:03.064997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:03.064997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:03.065429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:03.065429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:03.065429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:03.065429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:03.065429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.065429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:03.065429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.065429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:03.065429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:03.065429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:03.065429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:03.065429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:03.065429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.065429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:03.065841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:03.065841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.065841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:03.065841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:03.065841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.065841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:03.065841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:03.065841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.065841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:20:03.065841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:03.065841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:03.065841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:03.065841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:03.065841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:03.065841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:03.065841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:03.065841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:03.065841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:03.065841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:03.065841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:03.065841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:03.066439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:03.066439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:03.066439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:03.066439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:03.066439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:03.066439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:03.066439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:03.066439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.066439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:03.785251 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:03.785251 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:03.785471 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:03.786633 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:03.787066 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:03.787305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.788210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:03.788210 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:03.788210 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:03.788210 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:03.788210 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:03.788210 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:03.788210 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.788210 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:03.790068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.790068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:03.790068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:03.790068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:03.790068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.790068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:20:03.790068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:03.790068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:03.790068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:03.790068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:03.790068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:03.790068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:03.790354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:03.790354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:03.790354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:03.790354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:03.790354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:03.790354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:03.790354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:03.790354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:03.790354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:03.790354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:03.790354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:03.790354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:03.790354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:03.790646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.790646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:03.791920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.791920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:03.791920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:03.791920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:03.791920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:03.791920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:03.791920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.791920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:03.791920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:03.791920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.791920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:03.791920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:03.791920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.791920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:03.791920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:03.791920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.792255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:20:03.792255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:03.792255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:03.792255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:03.792255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:03.792255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:03.792255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:03.792255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:03.792255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:03.792255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:03.792255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:03.792255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:03.792255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:03.792255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:03.792255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:03.792255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:03.792255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:03.792255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:03.792255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:03.792608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:03.792608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.792608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:03.827723 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:03.827723 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:03.827723 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:03.827723 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:20:03.827723 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:03.827723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.828811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:03.828811 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:03.828811 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:03.828811 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:03.828811 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:03.828811 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:03.828811 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.828811 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:03.829632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.829632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:20:03.829632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:03.829632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:03.829632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:03.829632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:03.829632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:03.829632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:03.829632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:03.829632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:03.829632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:03.829632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:20:03.829632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:03.829632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:20:03.829632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:20:03.830115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:03.830115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:20:03.830115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:03.830115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:20:03.830115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:20:03.830115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:03.830115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:20:03.830115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.830115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:20:03.830539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.830539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:03.830539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:03.830539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:03.830539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.830539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:20:03.830539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:20:03.830539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:20:03.830539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:03.830539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:03.830539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:03.830539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:03.831230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:03.831230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:03.831230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:03.831230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:03.831230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:03.831230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:03.831230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:03.831230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:03.831230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:03.831230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:03.831230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:03.831230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:03.831230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:03.831230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:03.832111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:03.832111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:03.832111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.832111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:03.832111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.832111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:03.832111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:03.832111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:03.832111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:03.832111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:03.832111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.832111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:03.832111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:03.832111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.833199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:03.833199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:03.833199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.833199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:03.833199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:03.833199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.833199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:20:03.833199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:20:03.833199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:20:03.833199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:03.833199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:03.833199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:03.833199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:03.833199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:03.833199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:03.833199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:03.833199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:03.833199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:03.833199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:03.834123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:03.834123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:03.834123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:03.834123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:03.834123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:03.834123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:03.834123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:03.834123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:03.834123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:03.834123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:03.834123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.834123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:03.838345 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:03.838433 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:03.838473 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:03.838616 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:03.839230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:03.839582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.840346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:03.840346 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:03.840346 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:03.840346 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:03.840346 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:03.840346 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:03.840346 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.840346 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:03.841018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.841018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:20:03.841018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:03.841018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:03.841018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:03.841018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:03.841018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:03.841018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:03.841018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:03.841018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:03.841018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:03.841018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:20:03.841018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:03.841018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:20:03.841018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:20:03.841506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:03.841506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:20:03.841506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:03.841506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:20:03.841506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:20:03.841506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:03.841506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:20:03.841506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.841506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:20:03.841942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.841942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:03.841942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:03.841942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:03.841942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.841942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:20:03.841942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:20:03.841942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:03.841942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:03.841942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:03.841942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:03.842290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:03.842290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:03.842290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:03.842290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:03.842290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:03.842290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:03.842290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:03.842290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:03.842290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:03.842290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:03.842290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:03.842290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:03.842290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:03.842290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:03.842744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:03.842744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:03.842744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.842744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:03.842898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.842898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:03.842898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:03.842898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:03.842898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:03.842898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:03.842898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.842898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:03.842898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:03.842898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.842898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:03.842898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:03.842898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.842898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:03.842898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:03.842898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.842898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:20:03.843364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:20:03.843364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:03.843364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:03.843364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:03.843364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:03.843364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:03.843364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:03.843364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:03.843364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:03.843364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:03.843364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:03.843364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:03.843364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:03.843364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:03.843364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:03.843364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:03.843364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:03.843364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:03.843364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:03.844007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:03.844007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:03.844007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.844007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:03.847453 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:03.847453 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:03.847453 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:03.847597 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:03.848373 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:03.848804 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:03.849025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.850318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:03.850318 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:03.850318 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:03.850318 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:03.850318 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:03.850318 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:03.850318 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.850318 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:03.851218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.851218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:03.851218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:03.851218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:03.851218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.851218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:20:03.851218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:03.851218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:03.851218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:03.851218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:03.851218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:03.851218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:03.851702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:03.851702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:03.851702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:03.851702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:03.851702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:03.851702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:03.851702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:03.851702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:03.851702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:03.851702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:03.851702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:03.851702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:03.851702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:03.853001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.853001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:03.853001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.853001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:03.853001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:03.853001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:03.853001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:03.853001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:03.853001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.853001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:03.853001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:03.853001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.853001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:03.853001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:03.853001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.853444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:03.853444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:03.853444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:03.853444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:20:03.853444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:03.853444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:03.853444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:03.853444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:03.853444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:03.853444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:03.853444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:03.853444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:03.853444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:03.853444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:03.853444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:03.853444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:03.853444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:03.853444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:03.853444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:03.854020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:03.854020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:03.854020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:03.854020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:03.854020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:03.854020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:08.027407 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:08.028257 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:08.028257 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:08.028505 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:08.071860 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:08.072183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.073156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:08.073156 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:08.073156 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:08.073156 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:08.073156 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:08.073156 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:08.073156 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.073156 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:08.074440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.074440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:08.074440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:08.074440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:08.074440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.074440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:20:08.074440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:08.074440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:08.074440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:08.074440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:08.074440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:08.074440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:08.074870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:08.074870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:08.074870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:08.074870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:08.074870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:08.074870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:08.074870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:08.074870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:08.074870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:08.074870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:08.074870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:08.074870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:08.074870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:08.075364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.075364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:08.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:08.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:08.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:08.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:08.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:08.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:08.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:08.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:08.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:08.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:08.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:08.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.076502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:20:08.076502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:08.076502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:08.076502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:08.076502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:08.076502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:08.076502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:08.076502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:08.076502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:08.076502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:08.076502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:08.076502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:08.076502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:08.076502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:08.076502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:08.076502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:08.076502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:08.076502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:08.076502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:08.077063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:08.077063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.077063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:08.785387 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:08.786330 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:08.786533 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:08.787853 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:08.788091 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:08.788218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.788975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:08.788975 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:08.788975 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:08.788975 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:08.788975 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:08.788975 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:08.788975 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.788975 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:08.790672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.790672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:08.790672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:08.790672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:08.790672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.790672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:20:08.790672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:08.790672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:08.790672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:08.790672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:08.790672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:08.790672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:08.791104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:08.791104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:08.791104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:08.791104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:08.791104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:08.791104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:08.791104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:08.791104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:08.791104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:08.791104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:08.791104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:08.791104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:08.791104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:08.791429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.791429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:08.792366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.792366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:08.792366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:08.792366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:08.792366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:08.792366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:08.792366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.792366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:08.792366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:08.792366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.792366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:08.792366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:08.792366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.792366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:08.792366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:08.792366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.792914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:20:08.792914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:08.792914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:08.792914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:08.792914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:08.792914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:08.792914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:08.792914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:08.792914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:08.792914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:08.792914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:08.792914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:08.792914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:08.792914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:08.792914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:08.792914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:08.792914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:08.792914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:08.792914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:08.793377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:08.793377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.793377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:08.825363 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:08.826236 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:08.826461 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:08.826686 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:20:08.827192 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:08.827370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.827962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:08.827962 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:08.827962 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:08.827962 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:08.827962 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:08.827962 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:08.827962 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.827962 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:08.829465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.829465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:20:08.829465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:08.829465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:08.829465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:08.829465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:08.829465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:08.829465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:08.829465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:08.829465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:08.829465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:08.829465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:20:08.829465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:08.829465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:20:08.829465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:20:08.829961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:08.829961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:20:08.829961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:08.829961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:20:08.829961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:20:08.829961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:08.829961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:20:08.829961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.829961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:20:08.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:08.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:08.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:08.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:20:08.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:20:08.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:20:08.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:08.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:08.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:08.830491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:08.830892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:08.830892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:08.830892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:08.830892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:08.830892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:08.830892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:08.830892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:08.830892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:08.830892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:08.830892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:08.830892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:08.830892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:08.830892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:08.830892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:08.831264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:08.831264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:08.831264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.831264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:08.831386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.831386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:08.831386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:08.831386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:08.831386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:08.831386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:08.831386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.831386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:08.831386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:08.831386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.831386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:08.831386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:08.831386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.831386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:08.831386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:08.831386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.832037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:20:08.832037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:20:08.832037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:20:08.832037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:08.832037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:08.832037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:08.832037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:08.832037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:08.832037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:08.832037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:08.832037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:08.832037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:08.832037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:08.832037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:08.832037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:08.832037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:08.832037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:08.832037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:08.832843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:08.832843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:08.832843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:08.832843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:08.832843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:08.832843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.832843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:08.838634 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:08.838942 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:08.839040 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:08.839184 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:08.839619 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:08.839864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.840465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:08.840465 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:08.840465 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:08.840465 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:08.840465 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:08.840465 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:08.840465 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.840465 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:08.841732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.841732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:20:08.841732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:08.841732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:08.841732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:08.841732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:08.841732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:08.841732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:08.841732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:08.841732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:08.841732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:08.841732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:20:08.841732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:08.841732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:20:08.841732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:20:08.842222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:08.842222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:20:08.842222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:08.842222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:20:08.842222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:20:08.842222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:08.842222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:20:08.842222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.842222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:20:08.843271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.843271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:08.843271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:08.843271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:08.843271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.843271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:20:08.843271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:20:08.843271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:08.843271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:08.843271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:08.843271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:08.843614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:08.843614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:08.843614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:08.843614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:08.843614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:08.843614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:08.843614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:08.843614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:08.843614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:08.843614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:08.843614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:08.843614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:08.843614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:08.843614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:08.844074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:08.844074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:08.844074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.844074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:08.845108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.845108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:08.845108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:08.845108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:08.845108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:08.845108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:08.845108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.845108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:08.845108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:08.845108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.845108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:08.845108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:08.845108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.845108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:08.845108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:08.845108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.845108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:20:08.845640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:20:08.845640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:08.845640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:08.845640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:08.845640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:08.845640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:08.845640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:08.845640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:08.845640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:08.845640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:08.845640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:08.845640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:08.845640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:08.845640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:08.845640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:08.845640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:08.845640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:08.845640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:08.845640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:08.846192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:08.846192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:08.846192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.846192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:08.847618 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:08.847960 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:08.848299 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:08.848842 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:08.852069 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:08.852153 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:08.852264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.853283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:08.853283 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:08.853283 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:08.853283 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:08.853283 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:08.853283 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:08.853283 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.853283 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:08.855103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.855103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:08.855103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:08.855103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:08.855103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.855103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:20:08.855103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:08.855103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:08.855103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:08.855103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:08.855103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:08.855103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:08.855386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:08.855386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:08.855386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:08.855386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:08.855386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:08.855386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:08.855386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:08.855386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:08.855386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:08.855386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:08.855386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:08.855386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:08.855386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:08.855628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.855628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:08.857528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.857528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:08.857528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:08.857528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:08.857528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:08.857528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:08.857528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.857528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:08.857528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:08.857528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.857528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:08.857528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:08.857528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.857528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:08.857528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:08.857528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:08.857966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:20:08.857966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:08.857966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:08.857966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:08.857966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:08.857966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:08.857966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:08.857966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:08.857966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:08.857966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:08.857966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:08.857966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:08.857966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:08.857966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:08.857966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:08.857966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:08.857966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:08.857966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:08.857966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:08.858329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:08.858329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:08.858329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:13.027953 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:13.028870 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:13.028870 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:13.030099 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:13.061793 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:13.062233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.063074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:13.063074 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:13.063074 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:13.063074 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:13.063074 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:13.063074 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:13.063074 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.063074 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:13.063852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.063852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:13.063852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:13.063852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:13.063852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.063852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:20:13.063852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:13.063852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:13.063852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:13.063852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:13.063852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:13.063852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:13.064350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:13.064350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:13.064350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:13.064350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:13.064350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:13.064350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:13.064350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:13.064350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:13.064350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:13.064350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:13.064350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:13.064350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:13.064350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:13.064746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.064746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:13.064746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.064746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:13.064746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:13.064746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:13.064746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:13.064746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:13.064746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.064746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:13.064746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:13.064746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.064746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:13.064746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:13.064746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.065187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:13.065187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:13.065187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.065187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:20:13.065187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:13.065187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:13.065187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:13.065187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:13.065187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:13.065187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:13.065187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:13.065187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:13.065187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:13.065187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:13.065187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:13.065187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:13.065187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:13.065187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:13.065187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:13.065743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:13.065743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:13.065743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:13.065743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:13.065743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.065743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:13.786154 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:13.787122 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:13.787243 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:13.788644 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:13.789113 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:13.789387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.790584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:13.790584 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:13.790584 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:13.790584 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:13.790584 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:13.790584 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:13.790584 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.790584 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:13.791880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.791880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:13.791880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:13.791880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:13.791880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.791880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:20:13.791880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:13.791880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:13.791880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:13.791880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:13.791880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:13.791880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:13.792523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:13.792523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:13.792523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:13.792523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:13.792523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:13.792523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:13.792523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:13.792523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:13.792523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:13.792523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:13.792523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:13.792523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:13.792523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:13.793208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.793208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:13.793208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.793208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:13.793208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:13.793208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:13.793208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:13.793208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:13.793208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.793208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:13.793208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:13.793208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.793208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:13.793208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:13.793208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.793903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:13.793903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:13.793903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.793903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:20:13.793903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:13.793903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:13.793903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:13.793903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:13.793903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:13.793903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:13.793903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:13.793903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:13.793903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:13.793903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:13.793903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:13.793903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:13.793903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:13.793903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:13.793903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:13.794919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:13.794919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:13.794919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:13.794919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:13.794919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.794919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:13.826198 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:13.827131 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:13.827280 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:13.827478 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:20:13.828122 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:13.828362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.829111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:13.829111 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:13.829111 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:13.829111 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:13.829111 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:13.829111 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:13.829111 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.829111 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:13.830271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.830271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:20:13.830271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:13.830271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:13.830271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:13.830271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:13.830271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:13.830271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:13.830271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:13.830271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:13.830271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:13.830271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:20:13.830271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:13.830271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:20:13.830271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:20:13.830672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:13.830672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:20:13.830672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:13.830672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:20:13.830672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:20:13.830672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:13.830672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:20:13.830672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.830672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:20:13.831209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.831209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:13.831209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:13.831209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:13.831209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.831209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:20:13.831209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:20:13.831209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:20:13.831209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:13.831209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:13.831209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:13.831209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:13.831557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:13.831557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:13.831557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:13.831557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:13.831557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:13.831557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:13.831557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:13.831557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:13.831557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:13.831557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:13.831557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:13.831557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:13.831557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:13.831557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:13.831949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:13.831949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:13.831949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.831949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:13.832457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.832457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:13.832457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:13.832457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:13.832457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:13.832457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:13.832457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.832457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:13.832457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:13.832457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.832457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:13.832457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:13.832457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.832457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:13.832457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:13.832457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.832894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:20:13.832894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:20:13.832894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:20:13.832894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:13.832894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:13.832894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:13.832894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:13.832894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:13.832894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:13.832894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:13.832894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:13.832894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:13.832894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:13.832894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:13.832894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:13.832894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:13.832894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:13.832894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:13.833334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:13.833334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:13.833334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:13.833334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:13.833334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:13.833334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.833334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:13.839067 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:13.839507 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:13.839846 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:13.839846 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:13.840189 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:13.840418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.841049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:13.841049 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:13.841049 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:13.841049 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:13.841049 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:13.841049 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:13.841049 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.841049 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:13.841938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.841938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:20:13.841938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:13.841938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:13.841938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:13.841938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:13.841938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:13.841938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:13.841938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:13.841938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:13.841938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:13.841938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:20:13.841938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:13.841938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:20:13.841938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:20:13.842605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:13.842605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:20:13.842605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:13.842605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:20:13.842605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:20:13.842605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:13.842605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:20:13.842605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.842605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:20:13.843061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.843061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:13.843061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:13.843061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:13.843061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.843061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:20:13.843061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:20:13.843061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:13.843061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:13.843061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:13.843061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:13.843698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:13.843698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:13.843698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:13.843698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:13.843698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:13.843698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:13.843698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:13.843698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:13.843698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:13.843698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:13.843698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:13.843698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:13.843698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:13.843698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:13.844509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:13.844509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:13.844509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.844509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:13.844509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.844509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:13.844509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:13.844509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:13.844509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:13.844509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:13.844509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.844509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:13.844509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:13.844509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.844963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:13.844963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:13.844963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.844963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:13.844963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:13.844963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.844963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:20:13.844963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:20:13.844963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:13.844963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:13.844963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:13.844963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:13.844963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:13.844963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:13.844963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:13.844963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:13.844963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:13.844963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:13.844963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:13.845618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:13.845618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:13.845618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:13.845618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:13.845618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:13.845618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:13.845618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:13.845618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:13.845618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:13.845618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.845618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:13.847943 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:13.848271 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:13.848403 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:13.848600 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:13.849342 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:13.849745 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:13.849987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.850759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:13.850759 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:13.850759 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:13.850759 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:13.850759 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:13.850759 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:13.850759 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.850759 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:13.851825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.851825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:13.851825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:13.851825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:13.851825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.851825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:20:13.851825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:13.851825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:13.851825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:13.851825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:13.851825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:13.851825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:13.852180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:13.852180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:13.852180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:13.852180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:13.852180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:13.852180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:13.852180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:13.852180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:13.852180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:13.852180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:13.852180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:13.852180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:13.852180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:13.852572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.852572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:13.852901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.852901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:13.852901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:13.852901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:13.852901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:13.852901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:13.852901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.852901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:13.852901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:13.852901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.852901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:13.852901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:13.852901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.852901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:13.852901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:13.852901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:13.853394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:20:13.853394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:13.853394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:13.853394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:13.853394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:13.853394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:13.853394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:13.853394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:13.853394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:13.853394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:13.853394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:13.853394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:13.853394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:13.853394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:13.853394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:13.853394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:13.853394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:13.853394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:13.853394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:13.854015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:13.854015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:13.854015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:18.044710 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:18.044710 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:18.044710 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:18.046075 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:18.083185 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:18.083511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.085047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:18.085047 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:18.085047 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:18.085047 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:18.085047 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:18.085047 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:18.085047 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.085047 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:18.086773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.086773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:18.086773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:18.086773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:18.086773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.086773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:20:18.086773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:18.086773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:18.086773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:18.086773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:18.086773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:18.086773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:18.087256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:18.087256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:18.087256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:18.087256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:18.087256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:18.087256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:18.087256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:18.087256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:18.087256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:18.087256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:18.087256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:18.087256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:18.087256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:18.087600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.087600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:18.087600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.087600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:18.087600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:18.087600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:18.087600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:18.087600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:18.087600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.087600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:18.087600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:18.087600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.087600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:18.087600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:18.087600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.088019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:18.088019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:18.088019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.088019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:20:18.088019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:18.088019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:18.088019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:18.088019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:18.088019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:18.088019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:18.088019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:18.088019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:18.088019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:18.088019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:18.088019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:18.088019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:18.088019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:18.088019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:18.088019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:18.088482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:18.088482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:18.088482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:18.088482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:18.088482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.088482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:18.786994 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:18.787550 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:18.787776 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:18.788814 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:18.788998 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:18.789122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.789807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:18.789807 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:18.789807 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:18.789807 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:18.789807 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:18.789807 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:18.789807 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.789807 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:18.791351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.791351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:18.791351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:18.791351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:18.791351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.791351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:20:18.791351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:18.791351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:18.791351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:18.791351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:18.791351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:18.791351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:18.791692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:18.791692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:18.791692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:18.791692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:18.791692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:18.791692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:18.791692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:18.791692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:18.791692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:18.791692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:18.791692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:18.791692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:18.791692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:18.791942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.791942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:18.793246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.793246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:18.793246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:18.793246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:18.793246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:18.793246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:18.793246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.793246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:18.793246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:18.793246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.793246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:18.793246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:18.793246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.793246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:18.793246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:18.793246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.793582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:20:18.793582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:18.793582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:18.793582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:18.793582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:18.793582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:18.793582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:18.793582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:18.793582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:18.793582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:18.793582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:18.793582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:18.793582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:18.793582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:18.793582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:18.793582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:18.793582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:18.793582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:18.793582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:18.794114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:18.794114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.794114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:18.826912 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:18.827331 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:18.827531 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:18.827775 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:20:18.828327 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:18.828475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.829096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:18.829096 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:18.829096 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:18.829096 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:18.829096 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:18.829096 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:18.829096 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.829096 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:18.830383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.830383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:20:18.830383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:18.830383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:18.830383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:18.830383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:18.830383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:18.830383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:18.830383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:18.830383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:18.830383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:18.830383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:20:18.830383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:18.830383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:20:18.830383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:20:18.831222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:18.831222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:20:18.831222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:18.831222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:20:18.831222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:20:18.831222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:18.831222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:20:18.831222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.831222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:20:18.831872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.831872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:18.831872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:18.831872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:18.831872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.831872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:20:18.831872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:20:18.831872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:20:18.831872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:18.831872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:18.831872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:18.831872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:18.832210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:18.832210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:18.832210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:18.832210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:18.832210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:18.832210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:18.832210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:18.832210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:18.832210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:18.832210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:18.832210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:18.832210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:18.832210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:18.832210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:18.832580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:18.832580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:18.832580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.832580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:18.834003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.834003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:18.834003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:18.834003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:18.834003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:18.834003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:18.834003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.834003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:18.834003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:18.834003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.834003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:18.834003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:18.834003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.834003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:18.834003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:18.834003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:20:18.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:20:18.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:20:18.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:18.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:18.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:18.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:18.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:18.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:18.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:18.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:18.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:18.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:18.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:18.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:18.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:18.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:18.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:18.835149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:18.835149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:18.835149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:18.835149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:18.835149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:18.835149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.835149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:18.839517 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:18.839920 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:18.840098 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:18.840233 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:18.840612 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:18.840841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.841297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:18.841297 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:18.841297 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:18.841297 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:18.841297 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:18.841297 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:18.841297 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.841297 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:18.842215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.842215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:20:18.842215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:18.842215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:18.842215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:18.842215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:18.842215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:18.842215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:18.842215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:18.842215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:18.842215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:18.842215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:20:18.842215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:18.842215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:20:18.842215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:20:18.842643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:18.842643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:20:18.842643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:18.842643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:20:18.842643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:20:18.842643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:18.842643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:20:18.842643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.842643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:20:18.843149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.843149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:18.843149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:18.843149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:18.843149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.843149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:20:18.843149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:20:18.843149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:18.843149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:18.843149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:18.843149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:18.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:18.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:18.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:18.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:18.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:18.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:18.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:18.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:18.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:18.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:18.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:18.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:18.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:18.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:18.843850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:18.843850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:18.843850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.843850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:18.844361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.844361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:18.844361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:18.844361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:18.844361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:18.844361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:18.844361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.844361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:18.844361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:18.844361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.844361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:18.844361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:18.844361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.844361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:18.844361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:18.844361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.844361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:20:18.845312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:20:18.845312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:18.845312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:18.845312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:18.845312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:18.845312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:18.845312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:18.845312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:18.845312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:18.845312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:18.845312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:18.845312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:18.845312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:18.845312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:18.845312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:18.845312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:18.845312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:18.845312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:18.845312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:18.845845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:18.845845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:18.845845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.845845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:18.850150 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:18.850645 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:18.850791 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:18.850909 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:18.851249 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:18.851453 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:18.851569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.852029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:18.852029 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:18.852029 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:18.852029 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:18.852029 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:18.852029 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:18.852029 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.852029 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:18.852920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.852920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:18.852920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:18.852920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:18.852920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.852920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:20:18.852920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:18.852920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:18.852920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:18.852920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:18.852920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:18.852920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:18.853770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:18.853770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:18.853770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:18.853770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:18.853770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:18.853770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:18.853770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:18.853770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:18.853770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:18.853770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:18.853770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:18.853770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:18.853770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:18.854736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.854736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:18.854736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.854736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:18.854736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:18.854736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:18.854736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:18.854736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:18.854736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.854736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:18.854736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:18.854736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.854736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:18.854736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:18.854736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.855424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:18.855424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:18.855424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:18.855424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:20:18.855424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:18.855424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:18.855424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:18.855424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:18.855424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:18.855424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:18.855424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:18.855424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:18.855424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:18.855424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:18.855424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:18.855424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:18.855424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:18.855424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:18.855424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:18.856348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:18.856348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:18.856348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:18.856348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:18.856348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:18.856348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:23.044335 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:23.045210 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:23.045501 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:23.046863 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:23.079154 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:23.079417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.080193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:23.080193 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:23.080193 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:23.080193 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:23.080193 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:23.080193 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:23.080193 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.080193 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:23.080960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.080960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:23.080960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:23.080960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:23.080960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.080960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:20:23.080960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:23.080960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:23.080960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:23.080960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:23.080960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:23.080960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:23.081532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:23.081532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:23.081532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:23.081532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:23.081532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:23.081532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:23.081532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:23.081532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:23.081532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:23.081532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:23.081532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:23.081532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:23.081532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:23.082146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.082146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:23.082146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.082146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:23.082146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:23.082146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:23.082146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:23.082146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:23.082146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.082146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:23.082146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:23.082146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.082146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:23.082146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:23.082146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.082930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:23.082930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:23.082930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.082930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:20:23.082930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:23.082930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:23.082930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:23.082930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:23.082930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:23.082930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:23.082930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:23.082930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:23.082930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:23.082930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:23.082930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:23.082930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:23.082930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:23.082930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:23.082930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:23.083840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:23.083840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:23.083840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:23.083840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:23.083840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.083840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:23.787615 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:23.788349 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:23.788501 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:23.789881 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:23.790198 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:23.790451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.791284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:23.791284 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:23.791284 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:23.791284 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:23.791284 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:23.791284 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:23.791284 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.791284 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:23.792371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.792371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:23.792371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:23.792371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:23.792371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.792371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:20:23.792371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:23.792371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:23.792371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:23.792371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:23.792371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:23.792371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:23.792973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:23.792973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:23.792973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:23.792973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:23.792973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:23.792973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:23.792973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:23.792973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:23.792973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:23.792973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:23.792973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:23.792973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:23.792973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:23.793465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.793465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:23.793465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.793465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:23.793465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:23.793465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:23.793465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:23.793465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:23.793465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.793465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:23.793465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:23.793465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.793465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:23.793465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:23.793465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.794278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:23.794278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:23.794278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.794278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:20:23.794278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:23.794278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:23.794278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:23.794278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:23.794278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:23.794278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:23.794278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:23.794278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:23.794278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:23.794278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:23.794278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:23.794278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:23.794278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:23.794278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:23.794278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:23.795017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:23.795017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:23.795017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:23.795017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:23.795017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.795017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:23.827591 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:23.828440 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:23.828699 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:23.828938 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:20:23.829663 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:23.830003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.830835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:23.830835 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:23.830835 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:23.830835 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:23.830835 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:23.830835 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:23.830835 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.830835 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:23.832314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.832314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:20:23.832314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:23.832314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:23.832314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:23.832314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:23.832314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:23.832314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:23.832314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:23.832314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:23.832314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:23.832314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:20:23.832314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:23.832314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:20:23.832314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:20:23.832793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:23.832793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:20:23.832793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:23.832793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:20:23.832793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:20:23.832793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:23.832793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:20:23.832793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.832793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:20:23.833929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.833929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:23.833929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:23.833929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:23.833929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.833929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:20:23.833929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:20:23.833929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:20:23.833929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:23.833929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:23.833929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:23.833929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:23.834533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:23.834533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:23.834533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:23.834533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:23.834533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:23.834533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:23.834533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:23.834533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:23.834533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:23.834533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:23.834533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:23.834533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:23.834533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:23.834533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:23.835124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:23.835124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:23.835124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.835124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:23.836194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.836194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:23.836194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:23.836194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:23.836194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:23.836194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:23.836194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.836194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:23.836194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:23.836194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.836194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:23.836194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:23.836194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.836194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:23.836194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:23.836194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.836953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:20:23.836953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:20:23.836953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:20:23.836953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:23.836953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:23.836953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:23.836953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:23.836953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:23.836953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:23.836953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:23.836953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:23.836953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:23.836953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:23.836953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:23.836953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:23.836953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:23.836953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:23.836953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:23.837775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:23.837775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:23.837775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:23.837775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:23.837775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:23.837775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.837775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:23.840045 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:23.840338 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:23.840450 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:23.840596 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:23.841058 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:23.841414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.842114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:23.842114 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:23.842114 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:23.842114 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:23.842114 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:23.842114 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:23.842114 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.842114 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:23.842869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.842869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:20:23.842869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:23.842869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:23.842869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:23.842869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:23.842869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:23.842869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:23.842869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:23.842869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:23.842869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:23.842869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:20:23.842869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:23.842869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:20:23.842869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:20:23.843296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:23.843296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:20:23.843296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:23.843296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:20:23.843296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:20:23.843296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:23.843296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:20:23.843296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.843296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:20:23.843784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.843784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:23.843784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:23.843784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:23.843784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.843784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:20:23.843784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:20:23.843784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:23.843784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:23.843784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:23.843784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:23.844141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:23.844141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:23.844141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:23.844141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:23.844141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:23.844141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:23.844141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:23.844141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:23.844141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:23.844141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:23.844141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:23.844141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:23.844141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:23.844141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:23.844511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:23.844511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:23.844511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.844511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:23.844874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.844874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:23.844874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:23.844874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:23.844874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:23.844874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:23.844874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.844874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:23.844874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:23.844874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.844874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:23.844874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:23.844874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.844874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:23.844874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:23.844874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.844874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:20:23.845630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:20:23.845630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:23.845630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:23.845630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:23.845630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:23.845630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:23.845630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:23.845630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:23.845630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:23.845630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:23.845630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:23.845630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:23.845630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:23.845630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:23.845630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:23.845630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:23.845630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:23.845630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:23.845630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:23.846142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:23.846142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:23.846142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.846142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:23.850693 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:23.850941 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:23.851057 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:23.851213 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:23.851794 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:23.852197 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:23.852416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.853216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:23.853216 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:23.853216 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:23.853216 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:23.853216 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:23.853216 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:23.853216 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.853216 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:23.854126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.854126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:23.854126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:23.854126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:23.854126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.854126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:20:23.854126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:23.854126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:23.854126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:23.854126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:23.854126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:23.854126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:23.854476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:23.854476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:23.854476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:23.854476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:23.854476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:23.854476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:23.854476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:23.854476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:23.854476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:23.854476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:23.854476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:23.854476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:23.854476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:23.854840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.854840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:23.855924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.855924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:23.855924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:23.855924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:23.855924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:23.855924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:23.855924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.855924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:23.855924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:23.855924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.855924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:23.855924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:23.855924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.855924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:23.855924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:23.855924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:23.856385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:20:23.856385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:23.856385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:23.856385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:23.856385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:23.856385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:23.856385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:23.856385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:23.856385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:23.856385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:23.856385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:23.856385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:23.856385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:23.856385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:23.856385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:23.856385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:23.856385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:23.856385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:23.856385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:23.856888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:23.856888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:23.856888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:28.045161 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:28.045828 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:28.046184 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:28.046840 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:28.072696 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:28.073074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.073949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:28.073949 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:28.073949 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:28.073949 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:28.073949 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:28.073949 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:28.073949 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.073949 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:28.074774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.028 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.074774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:28.074774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:28.074774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:28.074774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.074774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:20:28.074774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:28.074774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:28.074774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:28.074774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:28.074774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:28.074774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:28.075969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:28.075969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:28.075969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:28.075969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:28.075969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:28.075969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:28.075969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:28.075969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:28.075969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:28.075969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:28.075969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:28.075969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:28.075969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:28.076583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.076583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:28.076583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.076583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:28.076583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:28.076583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:28.076583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:28.076583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:28.076583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.076583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:28.076583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:28.076583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.076583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:28.076583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:28.076583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.077247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:28.077247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:28.077247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.077247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:20:28.077247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:28.077247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:28.077247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:28.077247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:28.077247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:28.077247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:28.077247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:28.077247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:28.077247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:28.077247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:28.077247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:28.077247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:28.077247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:28.077247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:28.077247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:28.079093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:28.079093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:28.079093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:28.079093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:28.079093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.079093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:28.788627 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:28.789465 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:28.789633 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:28.790779 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:28.791091 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:28.791319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.792210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:28.792210 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:28.792210 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:28.792210 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:28.792210 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:28.792210 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:28.792210 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.792210 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:28.793843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.793843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:28.793843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:28.793843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:28.793843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.793843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:20:28.793843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:28.793843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:28.793843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:28.793843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:28.793843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:28.793843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:28.794371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:28.794371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:28.794371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:28.794371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:28.794371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:28.794371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:28.794371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:28.794371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:28.794371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:28.794371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:28.794371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:28.794371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:28.794371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:28.794773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.794773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:28.795714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.795714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:28.795714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:28.795714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:28.795714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:28.795714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:28.795714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.795714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:28.795714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:28.795714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.795714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:28.795714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:28.795714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.795714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:28.795714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:28.795714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:20:28.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:28.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:28.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:28.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:28.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:28.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:28.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:28.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:28.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:28.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:28.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:28.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:28.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:28.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:28.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:28.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:28.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:28.796208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:28.796892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:28.796892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.796892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:28.828271 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:28.828902 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:28.829058 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:28.829245 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:20:28.829845 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:28.830052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.830689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:28.830689 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:28.830689 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:28.830689 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:28.830689 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:28.830689 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:28.830689 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.830689 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:28.832064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.832064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:20:28.832064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:28.832064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:28.832064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:28.832064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:28.832064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:28.832064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:28.832064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:28.832064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:28.832064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:28.832064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:20:28.832064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:28.832064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:20:28.832064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:20:28.833364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:28.833364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:20:28.833364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:28.833364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:20:28.833364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:20:28.833364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:28.833364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:20:28.833364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.833364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:20:28.833364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.833364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:28.833364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:28.833364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:28.833364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.834080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:20:28.834080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:20:28.834080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:20:28.834080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:28.834080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:28.834080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:28.834080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:28.834080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:28.834080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:28.834080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:28.834080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:28.834080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:28.834080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:28.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:28.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:28.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:28.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:28.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:28.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:28.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:28.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:28.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:28.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:28.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.834637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:28.835179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.835179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:28.835179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:28.835179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:28.835179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:28.835179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:28.835179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.835179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:28.835179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:28.835179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.835179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:28.835179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:28.835179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.835179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:28.835179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:28.835179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.836160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:20:28.836160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:20:28.836160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:20:28.836160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:28.836160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:28.836160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:28.836160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:28.836160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:28.836160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:28.836160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:28.836160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:28.836160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:28.836160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:28.836160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:28.836160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:28.836160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:28.836160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:28.836160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:28.837078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:28.837078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:28.837078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:28.837078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:28.837078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:28.837078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.837078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:28.840398 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:28.840716 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:28.840860 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:28.841001 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:28.841501 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:28.841805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.842493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:28.842493 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:28.842493 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:28.842493 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:28.842493 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:28.842493 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:28.842493 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.842493 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:28.843985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.843985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:20:28.843985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:28.843985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:28.843985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:28.843985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:28.843985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:28.843985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:28.843985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:28.843985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:28.843985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:28.843985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:20:28.843985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:28.843985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:20:28.843985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:20:28.844284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:28.844284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:20:28.844284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:28.844284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:20:28.844284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:20:28.844284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:28.844284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:20:28.844284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.844284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:20:28.845601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.845601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:28.845601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:28.845601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:28.845601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.845601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:20:28.845601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:20:28.845601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:28.845601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:28.845601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:28.845601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:28.845854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:28.845854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:28.845854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:28.845854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:28.845854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:28.845854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:28.845854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:28.845854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:28.845854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:28.845854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:28.845854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:28.845854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:28.845854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:28.845854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:28.846101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:28.846101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:28.846101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.846101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:28.847698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.847698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:28.847698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:28.847698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:28.847698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:28.847698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:28.847698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.847698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:28.847698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:28.847698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.847698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:28.847698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:28.847698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.847698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:28.847698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:28.847698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.847698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:20:28.848455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:20:28.848455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:28.848455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:28.848455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:28.848455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:28.848455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:28.848455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:28.848455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:28.848455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:28.848455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:28.848455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:28.848455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:28.848455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:28.848455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:28.848455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:28.848455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:28.848455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:28.848455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:28.848455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:28.849364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:28.849364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:28.849364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.849364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:28.851091 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:28.851369 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:28.851479 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:28.851629 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:28.851980 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:28.852202 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:28.852349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.852897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:28.852897 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:28.852897 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:28.852897 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:28.852897 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:28.852897 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:28.852897 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.852897 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:28.853698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.853698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:28.853698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:28.853698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:28.853698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.853698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:20:28.853698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:28.853698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:28.853698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:28.853698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:28.853698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:28.853698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:28.854256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:28.854256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:28.854256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:28.854256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:28.854256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:28.854256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:28.854256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:28.854256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:28.854256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:28.854256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:28.854256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:28.854256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:28.854256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:28.854806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.854806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:28.854806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.854806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:28.854806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:28.854806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:28.854806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:28.854806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:28.854806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.854806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:28.854806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:28.854806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.854806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:28.854806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:28.854806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.855468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:28.855468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:28.855468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:28.855468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:20:28.855468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:28.855468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:28.855468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:28.855468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:28.855468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:28.855468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:28.855468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:28.855468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:28.855468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:28.855468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:28.855468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:28.855468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:28.855468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:28.855468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:28.855468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:28.856266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:28.856266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:28.856266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:28.856266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:28.856266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:28.856266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:33.045869 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:33.046606 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:33.046712 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:33.047958 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:33.075928 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:33.076166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.076769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:33.076769 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:33.076769 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:33.076769 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:33.076769 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:33.076769 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:33.076769 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.076769 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:33.078217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.078217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:33.078217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:33.078217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:33.078217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.078217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:20:33.078217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:33.078217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:33.078217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:33.078217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:33.078217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:33.078217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:33.079370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:33.079370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:33.079370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:33.079370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:33.079370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:33.079370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:33.079370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:33.079370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:33.079370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:33.079370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:33.079370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:33.079370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:33.079370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:33.080630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.080630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:33.080630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.080630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:33.080630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:33.080630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:33.080630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:33.080630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:33.080630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.080630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:33.080630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:33.080630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.080630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:33.080630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:33.080630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.081559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:33.081559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:33.081559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.081559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:20:33.081559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:33.081559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:33.081559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:33.081559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:33.081559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:33.081559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:33.081559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:33.081559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:33.081559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:33.081559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:33.081559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:33.081559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:33.081559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:33.081559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:33.081559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:33.082459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:33.082459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:33.082459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:33.082459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:33.082459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.082459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:33.789486 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:33.790173 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:33.790380 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:33.791477 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:33.791926 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:33.792168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.793104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:33.793104 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:33.793104 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:33.793104 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:33.793104 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:33.793104 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:33.793104 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.793104 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:33.794776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.794776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:33.794776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:33.794776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:33.794776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.794776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:20:33.794776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:33.794776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:33.794776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:33.794776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:33.794776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:33.794776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:33.795838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:33.795838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:33.795838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:33.795838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:33.795838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:33.795838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:33.795838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:33.795838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:33.795838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:33.795838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:33.795838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:33.795838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:33.795838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:33.796369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.796369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:33.796437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.796437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:33.796437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:33.796437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:33.796437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:33.796437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:33.796437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.796437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:33.796437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:33.796437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.796437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:33.796437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:33.796437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.796437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:33.796437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:33.796437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.796901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:20:33.796901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:33.796901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:33.796901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:33.796901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:33.796901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:33.796901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:33.796901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:33.796901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:33.796901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:33.796901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:33.796901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:33.796901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:33.796901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:33.796901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:33.796901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:33.796901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:33.796901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:33.796901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:33.797462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:33.797462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.797462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:33.829087 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:33.829426 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:33.829517 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:33.829667 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:20:33.830246 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:33.830574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.831420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:33.831420 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:33.831420 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:33.831420 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:33.831420 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:33.831420 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:33.831420 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.831420 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:33.833015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.833015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:20:33.833015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:33.833015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:33.833015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:33.833015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:33.833015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:33.833015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:33.833015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:33.833015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:33.833015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:33.833015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:20:33.833015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:33.833015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:20:33.833015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:20:33.833907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:33.833907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:20:33.833907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:33.833907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:20:33.833907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:20:33.833907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:33.833907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:20:33.833907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.833907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:20:33.835929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.835929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:33.835929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:33.835929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:33.835929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.835929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:20:33.835929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:20:33.835929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:20:33.835929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:33.835929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:33.835929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:33.835929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:33.836512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:33.836512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:33.836512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:33.836512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:33.836512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:33.836512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:33.836512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:33.836512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:33.836512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:33.836512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:33.836512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:33.836512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:33.836512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:33.836512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:33.837179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:33.837179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:33.837179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.837179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:33.837972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.837972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:33.837972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:33.837972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:33.837972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:33.837972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:33.837972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.837972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:33.837972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:33.837972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.837972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:33.837972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:33.837972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.837972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:33.837972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:33.837972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:20:33.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:20:33.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:20:33.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:33.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:33.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:33.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:33.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:33.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:33.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:33.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:33.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:33.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:33.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:33.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:33.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:33.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:33.838786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:33.839734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:33.839734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:33.839734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:33.839734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:33.839734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:33.839734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.839734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:33.840759 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:33.841139 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:33.841299 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:33.841443 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:33.841848 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:33.842086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.842584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:33.842584 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:33.842584 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:33.842584 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:33.842584 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:33.842584 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:33.842584 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.842584 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:33.843492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.843492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:20:33.843492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:33.843492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:33.843492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:33.843492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:33.843492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:33.843492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:33.843492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:33.843492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:33.843492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:33.843492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:20:33.843492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:33.843492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:20:33.843492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:20:33.843970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:33.843970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:20:33.843970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:33.843970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:20:33.843970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:20:33.843970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:33.843970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:20:33.843970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.843970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:20:33.844491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.844491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:33.844491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:33.844491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:33.844491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.844491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:20:33.844491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:20:33.844491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:33.844491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:33.844491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:33.844491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:33.845045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:33.845045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:33.845045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:33.845045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:33.845045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:33.845045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:33.845045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:33.845045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:33.845045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:33.845045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:33.845045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:33.845045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:33.845045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:33.845045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:33.845412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:33.845412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:33.845412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.845412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:33.845608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.845608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:33.845608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:33.845608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:33.845608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:33.845608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:33.845608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.845608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:33.845608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:33.845608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.845608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:33.845608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:33.845608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.845608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:33.845608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:33.845608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.845608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:20:33.846101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:20:33.846101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:33.846101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:33.846101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:33.846101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:33.846101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:33.846101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:33.846101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:33.846101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:33.846101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:33.846101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:33.846101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:33.846101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:33.846101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:33.846101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:33.846101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:33.846101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:33.846101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:33.846101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:33.846627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:33.846627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:33.846627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.846627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:33.851416 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:33.852598 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:33.852832 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:33.853128 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:33.853559 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:33.853841 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:33.853974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.854735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:33.854735 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:33.854735 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:33.854735 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:33.854735 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:33.854735 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:33.854735 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.854735 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:33.856239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.856239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:33.856239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:33.856239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:33.856239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.856239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:20:33.856239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:33.856239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:33.856239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:33.856239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:33.856239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:33.856239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:33.856621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:33.856621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:33.856621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:33.856621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:33.856621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:33.856621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:33.856621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:33.856621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:33.856621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:33.856621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:33.856621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:33.856621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:33.856621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:33.857054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.857054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:33.858376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.858376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:33.858376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:33.858376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:33.858376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:33.858376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:33.858376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.858376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:33.858376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:33.858376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.858376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:33.858376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:33.858376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.858376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:33.858376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:33.858376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:33.858710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:20:33.858710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:33.858710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:33.858710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:33.858710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:33.858710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:33.858710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:33.858710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:33.858710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:33.858710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:33.858710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:33.858710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:33.858710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:33.858710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:33.858710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:33.858710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:33.858710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:33.858710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:33.858710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:33.859083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:33.859083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:33.859083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:38.046566 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:38.050786 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:38.050786 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:38.051004 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:38.083408 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:38.083743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.084546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:38.084546 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:38.084546 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:38.084546 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:38.084546 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:38.084546 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:38.084546 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.084546 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:38.085578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.085578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:38.085578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:38.085578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:38.085578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.085578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:20:38.085578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:38.085578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:38.085578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:38.085578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:38.085578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:38.085578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:38.086402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:38.086402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:38.086402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:38.086402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:38.086402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:38.086402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:38.086402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:38.086402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:38.086402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:38.086402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:38.086402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:38.086402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:38.086402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:38.087218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.087218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:38.087218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.087218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:38.087218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:38.087218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:38.087218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:38.087218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:38.087218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.087218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:38.087218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:38.087218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.087218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:38.087218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:38.087218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.088022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:38.088022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:38.088022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.088022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:20:38.088022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:38.088022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:38.088022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:38.088022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:38.088022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:38.088022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:38.088022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:38.088022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:38.088022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:38.088022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:38.088022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:38.088022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:38.088022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:38.088022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:38.088022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:38.090019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:38.090019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:38.090019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:38.090019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:38.090019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.090019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:38.790106 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:38.791974 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:38.791974 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:38.791974 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:38.792222 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:38.792422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.793360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:38.793360 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:38.793360 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:38.793360 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:38.793360 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:38.793360 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:38.793360 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.793360 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:38.795061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.795061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:38.795061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:38.795061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:38.795061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.795061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:20:38.795061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:38.795061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:38.795061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:38.795061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:38.795061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:38.795061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:38.795536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:38.795536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:38.795536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:38.795536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:38.795536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:38.795536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:38.795536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:38.795536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:38.795536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:38.795536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:38.795536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:38.795536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:38.795536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:38.795975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.795975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:38.797113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.797113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:38.797113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:38.797113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:38.797113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:38.797113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:38.797113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.797113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:38.797113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:38.797113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.797113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:38.797113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:38.797113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.797113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:38.797113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:38.797113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.797706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:20:38.797706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:38.797706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:38.797706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:38.797706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:38.797706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:38.797706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:38.797706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:38.797706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:38.797706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:38.797706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:38.797706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:38.797706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:38.797706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:38.797706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:38.797706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:38.797706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:38.797706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:38.797706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:38.798404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:38.798404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.798404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:38.832201 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:38.832832 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:38.832942 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:38.833086 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:20:38.833531 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:38.833761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.834293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:38.834293 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:38.834293 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:38.834293 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:38.834293 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:38.834293 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:38.834293 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.834293 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:38.835188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.835188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:20:38.835188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:38.835188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:38.835188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:38.835188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:38.835188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:38.835188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:38.835188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:38.835188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:38.835188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:38.835188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:20:38.835188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:38.835188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:20:38.835188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:20:38.835653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:38.835653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:20:38.835653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:38.835653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:20:38.835653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:20:38.835653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:38.835653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:20:38.835653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.835653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:20:38.836003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.836003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:38.836003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:38.836003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:38.836003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.836003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:20:38.836003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:20:38.836003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:20:38.836003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:38.836003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:38.836003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:38.836003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:38.836346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:38.836346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:38.836346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:38.836346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:38.836346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:38.836346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:38.836346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:38.836346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:38.836346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:38.836346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:38.836346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:38.836346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:38.836346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:38.836346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:38.836729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:38.836729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:38.836729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.836729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:38.837107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.837107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:38.837107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:38.837107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:38.837107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:38.837107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:38.837107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.837107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:38.837107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:38.837107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.837107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:38.837107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:38.837107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.837107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:38.837107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:38.837107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.837615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:20:38.837615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:20:38.837615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:20:38.837615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:38.837615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:38.837615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:38.837615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:38.837615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:38.837615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:38.837615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:38.837615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:38.837615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:38.837615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:38.837615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:38.837615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:38.837615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:38.837615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:38.837615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:38.838154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:38.838154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:38.838154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:38.838154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:38.838154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:38.838154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.838154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:38.841193 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:38.841580 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:38.841728 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:38.841876 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:38.842243 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:38.842504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.843071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:38.843071 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:38.843071 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:38.843071 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:38.843071 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:38.843071 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:38.843071 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.843071 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:38.843922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.843922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:20:38.843922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:38.843922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:38.843922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:38.843922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:38.843922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:38.843922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:38.843922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:38.843922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:38.843922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:38.843922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:20:38.843922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:38.843922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:20:38.843922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:20:38.844804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:38.844804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:20:38.844804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:38.844804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:20:38.844804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:20:38.844804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:38.844804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:20:38.844804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.844804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:20:38.845046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.845046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:38.845046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:38.845046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:38.845046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.845046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:20:38.845046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:20:38.845046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:38.845046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:38.845046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:38.845046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:38.845360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:38.845360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:38.845360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:38.845360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:38.845360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:38.845360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:38.845360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:38.845360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:38.845360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:38.845360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:38.845360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:38.845360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:38.845360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:38.845360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:38.845749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:38.845749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:38.845749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.845749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:38.846264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.846264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:38.846264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:38.846264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:38.846264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:38.846264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:38.846264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.846264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:38.846264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:38.846264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.846264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:38.846264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:38.846264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.846264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:38.846264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:38.846264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.846264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:20:38.846801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:20:38.846801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:38.846801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:38.846801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:38.846801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:38.846801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:38.846801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:38.846801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:38.846801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:38.846801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:38.846801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:38.846801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:38.846801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:38.846801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:38.846801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:38.846801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:38.846801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:38.846801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:38.846801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:38.847312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:38.847312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:38.847312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.847312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:38.852154 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:38.852713 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:38.852981 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:38.853355 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:38.853750 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:38.853979 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:38.854125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.854771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:38.854771 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:38.854771 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:38.854771 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:38.854771 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:38.854771 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:38.854771 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.854771 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:38.856360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.856360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:38.856360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:38.856360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:38.856360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.856360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:20:38.856360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:38.856360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:38.856360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:38.856360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:38.856360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:38.856360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:38.856721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:38.856721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:38.856721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:38.856721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:38.856721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:38.856721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:38.856721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:38.856721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:38.856721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:38.856721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:38.856721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:38.856721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:38.856721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:38.857064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.857064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:38.858157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.858157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:38.858157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:38.858157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:38.858157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:38.858157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:38.858157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.858157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:38.858157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:38.858157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.858157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:38.858157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:38.858157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.858157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:38.858157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:38.858157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:38.858655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:20:38.858655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:38.858655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:38.858655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:38.858655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:38.858655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:38.858655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:38.858655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:38.858655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:38.858655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:38.858655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:38.858655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:38.858655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:38.858655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:38.858655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:38.858655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:38.858655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:38.858655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:38.858655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:38.859201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:38.859201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:38.859201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:43.050189 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:43.050897 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:43.050897 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:43.051557 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:43.085571 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:43.085886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.086556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:43.086556 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:43.086556 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:43.086556 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:43.086556 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:43.086556 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:43.086556 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.086556 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:43.087324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.087324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:43.087324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:43.087324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:43.087324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.087324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:20:43.087324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:43.087324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:43.087324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:43.087324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:43.087324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:43.087324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:43.087976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:43.087976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:43.087976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:43.087976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:43.087976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:43.087976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:43.087976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:43.087976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:43.087976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:43.087976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:43.087976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:43.087976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:43.087976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:43.088572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.088572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:43.088572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.088572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:43.088572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:43.088572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:43.088572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:43.088572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:43.088572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.088572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:43.088572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:43.088572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.088572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:43.088572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:43.088572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.089249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:43.089249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:43.089249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.089249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:20:43.089249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:43.089249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:43.089249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:43.089249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:43.089249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:43.089249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:43.089249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:43.089249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:43.089249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:43.089249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:43.089249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:43.089249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:43.089249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:43.089249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:43.089249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:43.090097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:43.090097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:43.090097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:43.090097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:43.090097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.090097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:43.801649 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:43.801649 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:43.801649 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:43.801649 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:43.801649 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:43.801649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.802713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:43.802713 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:43.802713 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:43.802713 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:43.802713 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:43.802713 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:43.802713 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.802713 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:43.802713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.802713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:43.802713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:43.802713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:43.802713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.802713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:20:43.802713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:43.803722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:43.803722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:43.803722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:43.803722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:43.803722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:43.803722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:43.803722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:43.803722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:43.803722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:43.803722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:43.803722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:43.803722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:43.803722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:43.803722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:43.804283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:43.804283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:43.804283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:43.804283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:43.804283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.804283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:43.804283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.804283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:43.804283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:43.804283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:43.804283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:43.804283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:43.804283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.804283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:43.805494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:43.805494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.805494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:43.805494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:43.805494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.805494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:43.805494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:43.805494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.805494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:20:43.805494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:43.805494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:43.805494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:43.805494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:43.805494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:43.805494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:43.805494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:43.805494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:43.805494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:43.805494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:43.805494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:43.806295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:43.806295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:43.806295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:43.806295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:43.806295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:43.806295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:43.806295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:43.806295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:43.806295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.806295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:43.833006 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:43.833803 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:43.833957 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:43.834174 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:20:43.834820 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:43.835012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.836034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:43.836034 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:43.836034 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:43.836034 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:43.836034 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:43.836034 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:43.836034 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.836034 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:43.836997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.836997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:20:43.836997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:43.836997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:43.836997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:43.836997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:43.836997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:43.836997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:43.836997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:43.836997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:43.836997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:43.836997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:20:43.836997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:43.836997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:20:43.836997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:20:43.837567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:43.837567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:20:43.837567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:43.837567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:20:43.837567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:20:43.837567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:43.837567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:20:43.837567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.837567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:20:43.838219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.838219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:43.838219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:43.838219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:43.838219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.838219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:20:43.838219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:20:43.838219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:20:43.838219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:43.838219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:43.838219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:43.838219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:43.838760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:43.838760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:43.838760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:43.838760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:43.838760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:43.838760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:43.838760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:43.838760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:43.838760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:43.838760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:43.838760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:43.838760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:43.838760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:43.838760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:43.839262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:43.839262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:43.839262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.839262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:43.839860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.839860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:43.839860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:43.839860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:43.839860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:43.839860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:43.839860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.839860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:43.839860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:43.839860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.839860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:43.839860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:43.839860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.839860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:43.839860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:43.839860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.840351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:20:43.840351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:20:43.840351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:20:43.840351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:43.840351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:43.840351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:43.840351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:43.840351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:43.840351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:43.840351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:43.840351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:43.840351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:43.840351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:43.840351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:43.840351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:43.840351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:43.840351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:43.840351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:43.840864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:43.840864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:43.840864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:43.840864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:43.840864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:43.840864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.840864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:43.841645 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:43.841933 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:43.842196 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:43.842349 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:43.842728 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:43.842953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.843496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:43.843496 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:43.843496 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:43.843496 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:43.843496 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:43.843496 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:43.843496 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.843496 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:43.844375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.844375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:20:43.844375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:43.844375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:43.844375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:43.844375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:43.844375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:43.844375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:43.844375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:43.844375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:43.844375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:43.844375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:20:43.844375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:43.844375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:20:43.844375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:20:43.845052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:43.845052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:20:43.845052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:43.845052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:20:43.845052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:20:43.845052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:43.845052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:20:43.845052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.845052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:20:43.845690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.845690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:43.845690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:43.845690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:43.845690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.845690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:20:43.845690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:20:43.845690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:43.845690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:43.845690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:43.845690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:43.846145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:43.846145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:43.846145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:43.846145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:43.846145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:43.846145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:43.846145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:43.846145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:43.846145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:43.846145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:43.846145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:43.846145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:43.846145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:43.846145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:43.846652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:43.846652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:43.846652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.846652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:43.847934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.847934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:43.847934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:43.847934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:43.847934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:43.847934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:43.847934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.847934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:43.847934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:43.847934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.847934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:43.847934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:43.847934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.847934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:43.847934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:43.847934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.847934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:20:43.848520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:20:43.848520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:43.848520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:43.848520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:43.848520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:43.848520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:43.848520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:43.848520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:43.848520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:43.848520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:43.848520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:43.848520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:43.848520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:43.848520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:43.848520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:43.848520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:43.848520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:43.848520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:43.848520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:43.849231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:43.849231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:43.849231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.849231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:43.852595 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:43.852897 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:43.853063 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:43.853229 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:43.853545 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:43.853770 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:43.854298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.854788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:43.854788 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:43.854788 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:43.854788 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:43.854788 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:43.854788 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:43.854788 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.854788 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:43.855515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.855515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:43.855515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:43.855515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:43.855515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.855515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:20:43.855515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:43.855515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:43.855515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:43.855515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:43.855515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:43.855515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:43.855854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:43.855854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:43.855854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:43.855854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:43.855854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:43.855854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:43.855854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:43.855854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:43.855854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:43.855854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:43.855854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:43.855854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:43.855854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:43.856216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.856216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:43.856488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.856488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:43.856488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:43.856488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:43.856488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:43.856488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:43.856488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.856488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:43.856488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:43.856488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.856488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:43.856488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:43.856488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.856488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:43.856488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:43.856488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:43.856904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:20:43.856904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:43.856904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:43.856904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:43.856904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:43.856904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:43.856904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:43.856904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:43.856904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:43.856904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:43.856904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:43.856904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:43.856904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:43.856904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:43.856904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:43.856904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:43.856904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:43.856904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:43.856904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:43.857238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:43.857238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:43.857238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:48.050845 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:48.053735 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:48.053735 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:48.053735 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:48.084466 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:48.084976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.085757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:48.085757 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:48.085757 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:48.085757 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:48.085757 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:48.085757 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:48.085757 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.085757 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:48.088109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.088109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:48.088109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:48.088109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:48.088109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.088109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:20:48.088109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:48.088109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:48.088109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:48.088109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:48.088109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:48.088109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:48.088448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:48.088448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:48.088448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:48.088448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:48.088448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:48.088448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:48.088448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:48.088448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:48.088448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:48.088448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:48.088448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:48.088448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:48.088448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:48.088830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.088830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:48.088830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.088830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:48.088830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:48.088830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:48.088830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:48.088830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:48.088830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.088830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:48.088830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:48.088830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.088830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:48.088830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:48.088830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.089306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:48.089306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:48.089306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.089306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:20:48.089306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:48.089306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:48.089306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:48.089306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:48.089306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:48.089306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:48.089306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:48.089306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:48.089306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:48.089306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:48.089306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:48.089306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:48.089306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:48.089306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:48.089306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:48.089821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:48.089821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:48.089821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:48.089821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:48.089821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.089821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:48.791607 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:48.793081 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:48.793081 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:48.793274 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:48.793577 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:48.793851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.797721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:48.797721 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:48.797721 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:48.797721 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:48.797721 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:48.797721 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:48.797721 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.797721 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:48.797721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.797721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:48.797721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:48.797721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:48.797721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.797721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:20:48.797721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:48.798766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:48.798766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:48.798766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:48.798766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:48.798766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:48.798766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:48.798766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:48.798766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:48.798766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:48.798766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:48.798766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:48.798766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:48.798766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:48.798766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:48.799657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:48.799657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:48.799657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:48.799657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:48.799657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.799657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:48.800554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.800554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:48.800554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:48.800554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:48.800554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:48.800554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:48.800554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.800554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:48.800554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:48.800554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.800554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:48.800554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:48.800554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.800554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:48.800554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:48.800554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.801906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:20:48.801906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:48.801906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:48.801906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:48.801906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:48.801906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:48.801906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:48.801906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:48.801906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:48.801906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:48.801906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:48.801906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:48.801906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:48.801906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:48.801906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:48.801906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:48.801906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:48.801906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:48.801906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:48.802377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:48.802377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.802377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:48.834115 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:48.835274 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:48.835274 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:48.835274 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:20:48.835728 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:48.835871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.836830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:48.836830 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:48.836830 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:48.836830 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:48.836830 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:48.836830 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:48.836830 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.836830 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:48.838744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.838744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:20:48.838744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:48.838744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:48.838744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:48.838744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:48.838744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:48.838744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:48.838744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:48.838744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:48.838744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:48.838744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:20:48.838744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:48.838744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:20:48.838744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:20:48.839210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:48.839210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:20:48.839210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:48.839210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:20:48.839210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:20:48.839210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:48.839210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:20:48.839210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.839210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:20:48.840408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.840408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:48.840408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:48.840408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:48.840408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.840408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:20:48.840408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:20:48.840408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:20:48.840408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:48.840408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:48.840408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:48.840408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:48.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:48.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:48.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:48.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:48.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:48.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:48.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:48.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:48.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:48.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:48.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:48.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:48.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:48.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:48.841254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:48.841254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:48.841254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.841254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:48.842558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.842558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:48.842558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:48.842558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:48.842558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:48.842558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:48.842558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.842558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:48.842558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:48.842558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.842558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:48.842558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:48.842558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.842558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:48.842558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:48.842558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.843171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:20:48.843171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:20:48.843171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:20:48.843171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:48.843171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:48.843171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:48.843171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:48.843171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:48.843171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:48.843171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:48.843171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:48.843171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:48.843171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:48.843171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:48.843171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:48.843171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:48.843171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:48.843171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:48.843744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:48.843744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:48.843744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:48.843744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:48.843744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:48.843744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.843744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:48.843744 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:48.843744 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:48.843744 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:48.844110 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:48.844178 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:48.844426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.844888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:48.844888 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:48.844888 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:48.844888 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:48.844888 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:48.844888 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:48.844888 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.844888 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:48.845500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.845500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:20:48.845500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:48.845500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:48.845500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:48.845500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:48.845500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:48.845500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:48.845500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:48.845500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:48.845500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:48.845500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:20:48.845500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:48.845500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:20:48.845500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:20:48.845964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:48.845964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:20:48.845964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:48.845964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:20:48.845964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:20:48.845964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:48.845964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:20:48.845964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.845964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:20:48.846355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.846355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:48.846355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:48.846355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:48.846355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.846355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:20:48.846355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:20:48.846355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:48.846355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:48.846355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:48.846355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:48.846737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:48.846737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:48.846737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:48.846737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:48.846737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:48.846737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:48.846737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:48.846737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:48.846737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:48.846737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:48.846737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:48.846737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:48.846737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:48.846737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:48.847137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:48.847137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:48.847137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.847137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:48.847381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.847381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:48.847381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:48.847381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:48.847381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:48.847381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:48.847381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.847381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:48.847381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:48.847381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.847381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:48.847381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:48.847381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.847381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:48.847381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:48.847381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.847381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:20:48.847952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:20:48.847952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:48.847952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:48.847952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:48.847952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:48.847952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:48.847952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:48.847952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:48.847952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:48.847952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:48.847952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:48.847952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:48.847952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:48.847952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:48.847952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:48.847952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:48.847952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:48.847952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:48.847952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:48.848586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:48.848586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:48.848586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.848586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:48.852953 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:48.853210 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:48.853374 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:48.853573 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:48.854007 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:48.854369 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:48.854590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.855222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:48.855222 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:48.855222 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:48.855222 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:48.855222 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:48.855222 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:48.855222 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.855222 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:48.856189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.856189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:48.856189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:48.856189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:48.856189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.856189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:20:48.856189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:48.856189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:48.856189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:48.856189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:48.856189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:48.856189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:48.856521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:48.856521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:48.856521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:48.856521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:48.856521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:48.856521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:48.856521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:48.856521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:48.856521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:48.856521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:48.856521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:48.856521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:48.856521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:48.856886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.856886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:48.857550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.857550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:48.857550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:48.857550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:48.857550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:48.857550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:48.857550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.857550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:48.857550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:48.857550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.857550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:48.857550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:48.857550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.857550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:48.857550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:48.857550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:48.857999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:20:48.857999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:48.857999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:48.857999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:48.857999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:48.857999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:48.857999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:48.857999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:48.857999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:48.857999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:48.857999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:48.857999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:48.857999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:48.857999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:48.857999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:48.857999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:48.857999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:48.857999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:48.857999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:48.858499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:48.858499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:48.858499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:53.052123 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:53.052879 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:53.052879 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:53.053745 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:53.085613 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:53.086007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.086495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:53.086495 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:53.086495 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:53.086495 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:53.086495 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:53.086495 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:53.086495 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.086495 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:53.087489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.087489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:53.087489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:53.087489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:53.087489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.087489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:20:53.087489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:53.087489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:53.087489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:53.087489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:53.087489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:53.087489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:53.088142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:53.088142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:53.088142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:53.088142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:53.088142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:53.088142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:53.088142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:53.088142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:53.088142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:53.088142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:53.088142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:53.088142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:53.088142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:53.088885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.088885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:53.088885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.088885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:53.088885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:53.088885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:53.088885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:53.088885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:53.088885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.088885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:53.088885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:53.088885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.088885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:53.088885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:53.088885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.089590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:53.089590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:53.089590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.089590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:20:53.089590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:53.089590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:53.089590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:53.089590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:53.089590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:53.089590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:53.089590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:53.089590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:53.089590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:53.089590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:53.089590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:53.089590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:53.089590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:53.089590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:53.089590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:53.090516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:53.090516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:53.090516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:53.090516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:53.090516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.090516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:53.792316 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:53.792963 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:53.793170 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:53.794245 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:53.794565 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:53.794835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.795711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:53.795711 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:53.795711 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:53.795711 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:53.795711 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:53.795711 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:53.795711 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.795711 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:53.797203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.797203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:53.797203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:53.797203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:53.797203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.797203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:20:53.797203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:53.797203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:53.797203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:53.797203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:53.797203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:53.797203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:53.797628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:53.797628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:53.797628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:53.797628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:53.797628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:53.797628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:53.797628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:53.797628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:53.797628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:53.797628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:53.797628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:53.797628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:53.797628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:53.797919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.797919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:53.798861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.798861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:53.798861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:53.798861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:53.798861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:53.798861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:53.798861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.798861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:53.798861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:53.798861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.798861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:53.798861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:53.798861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.798861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:53.798861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:53.798861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.799401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:20:53.799401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:53.799401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:53.799401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:53.799401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:53.799401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:53.799401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:53.799401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:53.799401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:53.799401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:53.799401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:53.799401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:53.799401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:53.799401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:53.799401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:53.799401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:53.799401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:53.799401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:53.799401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:53.799951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:53.799951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.799951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:53.835256 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:53.835769 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:53.836014 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:53.836237 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:20:53.843431 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:53.872259 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:53.872696 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:53.872950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.873533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:53.873533 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:53.873533 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:53.873533 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:53.873533 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:53.873533 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:53.873533 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.873533 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:53.874140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.874140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:20:53.874140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:53.874140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:53.874140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:53.874140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:53.874140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:53.874140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:53.874140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:53.874140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:53.874140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:53.874140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:20:53.874140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:53.874140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:20:53.874140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:20:53.874590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:53.874590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:20:53.874590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:53.874590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:20:53.874590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:20:53.874590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:53.874590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:20:53.874590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.874590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:20:53.874898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.874898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:53.874898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:53.874898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:53.874898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.874898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:20:53.874898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:20:53.874898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:20:53.874898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:53.874898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:53.874898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:53.874898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:53.875228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:53.875228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:53.875228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:53.875228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:53.875228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:53.875228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:53.875228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:53.875228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:53.875228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:53.875228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:53.875228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:53.875228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:53.875228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:53.875228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:53.875656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:53.875656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:53.875656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.875656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:53.875656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.875656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:53.875656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:53.875656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:53.875656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:53.875656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:53.875656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.875656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:53.875656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:53.875656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.876183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:53.876183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:53.876183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.876183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:53.876183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:53.876183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.876183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:20:53.876183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:20:53.876183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:20:53.876183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:53.876183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:53.876183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:53.876183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:53.876183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:53.876183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:53.876183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:53.876183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:53.876183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:53.876183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:53.876597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:53.876597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:53.876597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:53.876597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:53.876597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:53.876597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:53.876597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:53.876597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:53.876597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:53.876597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:53.876597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.876597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:53.876597 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:53.876597 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:53.876938 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:53.876938 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:53.876938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.877151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:53.877151 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:53.877151 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:53.877151 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:53.877151 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:53.877151 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:53.877151 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.877151 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:53.878392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.878392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:20:53.878392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:53.878392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:53.878392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:53.878392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:53.878392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:53.878392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:53.878392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:53.878392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:53.878392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:53.878392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:20:53.878392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:53.878392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:20:53.878392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:20:53.878736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:53.878736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:20:53.878736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:53.878736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:20:53.878736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:20:53.878736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:53.878736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:20:53.878736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.878736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:20:53.879435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.879435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:53.879435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:53.879435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:53.879435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.879435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:20:53.879435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:20:53.879435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:53.879435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:53.879435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:53.879435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:53.879736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:53.879736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:53.879736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:53.879736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:53.879736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:53.879736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:53.879736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:53.879736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:53.879736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:53.879736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:53.879736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:53.879736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:53.879736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:53.879736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:53.880199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:53.880199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:53.880199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.880199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:53.880199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.880199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:53.880199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:53.880199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:53.880199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:53.880199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:53.880199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.880199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:53.880199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:53.880199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.880537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:53.880537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:53.880537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.880537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:53.880537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:53.880537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.880537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:20:53.880537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:20:53.880537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:53.880537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:53.880537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:53.880537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:53.880537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:53.880537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:53.880537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:53.880537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:53.880537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:53.880537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:53.880537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:53.880949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:53.880949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:53.880949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:53.880949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:53.880949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:53.880949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:53.880949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:53.880949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:53.880949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:53.880949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.880949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:53.880949 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:53.880949 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:53.881228 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:53.881228 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:53.881228 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:53.881228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.881357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:53.881357 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:53.881357 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:53.881357 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:53.881357 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:53.881357 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:53.881357 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.881357 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:53.882137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.882137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:53.882137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:53.882137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:53.882137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.882137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:20:53.882137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:53.882137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:53.882137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:53.882137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:53.882137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:53.882137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:53.882464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:53.882464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:53.882464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:53.882464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:53.882464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:53.882464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:53.882464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:53.882464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:53.882464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:53.882464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:53.882464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:53.882464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:53.882464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:53.882783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.882783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:53.883085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.883085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:53.883085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:53.883085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:53.883085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:53.883085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:53.883085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.883085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:53.883085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:53.883085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.883085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:53.883085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:53.883085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.883085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:53.883085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:53.883085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:53.883518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:20:53.883518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:53.883518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:53.883518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:53.883518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:53.883518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:53.883518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:53.883518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:53.883518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:53.883518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:53.883518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:53.883518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:53.883518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:53.883518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:53.883518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:53.883518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:53.883518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:53.883518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:53.883518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:53.884064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:53.884064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:53.884064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:58.052851 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:58.053415 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:58.053415 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:58.054248 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:58.093849 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:58.094032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.095338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:58.095338 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:58.095338 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:58.095338 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:58.095338 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:58.095338 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:58.095338 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.095338 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:58.096215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.096215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:58.096215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:58.096215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:58.096215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.096215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:20:58.096215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:58.096215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:58.096215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:58.096215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:58.096215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:58.096215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:58.096667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:58.096667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:58.096667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:58.096667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:58.096667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:58.096667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:58.096667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:58.096667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:58.096667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:58.096667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:58.096667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:58.096667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:58.096667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:58.097148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.097148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:58.097304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.097304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:58.097304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:58.097304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:58.097304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:58.097304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:58.097304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.097304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:58.097304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:58.097304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.097304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:58.097304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:58.097304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.097304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:58.097304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:58.097304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.097808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:20:58.097808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:58.097808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:58.097808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:58.097808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:58.097808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:58.097808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:58.097808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:58.097808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:58.097808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:58.097808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:58.097808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:58.097808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:58.097808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:58.097808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:58.097808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:58.097808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:58.097808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:58.097808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:58.098333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:58.098333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.098333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:58.793025 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:58.793591 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:58.793831 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:58.794700 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:58.794937 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:58.795133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.795862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:58.795862 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:58.795862 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:58.795862 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:58.795862 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:58.795862 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:58.795862 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.795862 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:58.796920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.796920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:58.796920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:58.796920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:58.796920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.796920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:20:58.796920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:58.796920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:58.796920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:58.796920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:58.796920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:58.796920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:58.797464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:58.797464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:58.797464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:58.797464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:58.797464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:58.797464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:58.797464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:58.797464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:58.797464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:58.797464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:58.797464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:58.797464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:58.797464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:58.798046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.798046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:58.798046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.798046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:58.798046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:58.798046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:58.798046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:58.798046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:58.798046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.798046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:58.798046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:58.798046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.798046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:58.798046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:58.798046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.798710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:58.798710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:58.798710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.798710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:20:58.798710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:58.798710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:58.798710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:58.798710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:58.798710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:58.798710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:58.798710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:58.798710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:58.798710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:58.798710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:58.798710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:58.798710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:58.798710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:58.798710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:58.798710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:58.799521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:58.799521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:58.799521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:58.799521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:58.799521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.799521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:58.835946 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:58.836636 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:58.836891 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:58.837096 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:20:58.840714 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:58.840714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.840714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:58.840714 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:58.840714 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:58.840714 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:58.840714 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:58.840714 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:58.840714 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.840714 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:58.840714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.840714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:20:58.840714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:58.840714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:58.841411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:58.841411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:58.841411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:58.841411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:58.841411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:58.841411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:58.841411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:58.841411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:20:58.841411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:58.841411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:20:58.841411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:20:58.841411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:58.841411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:20:58.841411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:58.841411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:20:58.841411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:20:58.841411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:58.841411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:20:58.842040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.842040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:20:58.842040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.842040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:58.842040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:58.842040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:58.842040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.842040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:20:58.842040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:20:58.842040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:20:58.842040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:58.842040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:58.842398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:58.842398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:58.842398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:58.842398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:58.842398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:58.842398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:58.842398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:58.842398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:58.842398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:58.842398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:58.842398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:58.842398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:58.842398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:58.842398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:58.842819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:58.842819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:58.842819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:58.842819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:58.842819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.842819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:58.842819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.842819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:58.842819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:58.842819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:58.842819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:58.842819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:58.842819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.842819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:58.843218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:58.843218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.843218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:58.843218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:58.843218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.843218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:58.843218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:58.843218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.843218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:20:58.843218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:20:58.843218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:20:58.843218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:58.843218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:58.843218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:58.843218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:58.843218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:58.843218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:58.843218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:58.843218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:58.843780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:58.843780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:58.843780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:58.843780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:58.843780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:58.843780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:58.843780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:58.843780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:58.843780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:58.843780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:58.843780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:58.843780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:58.843780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.843780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:58.876212 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:58.876772 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:58.876974 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:58.877215 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:58.877946 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:58.878411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.880120 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:20:58.880729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:58.880729 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:58.880729 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:58.880729 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:58.880729 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:58.880729 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:58.880729 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.880729 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:58.885376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.885376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:20:58.885376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:58.885376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:58.885376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:58.885376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:58.885376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:58.885376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:58.885376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:58.885376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:20:58.885376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:58.885376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:20:58.885376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:58.885376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:20:58.885376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:20:58.886105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:58.886105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:20:58.886105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:58.886105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:20:58.886105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:20:58.886105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:58.886105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:20:58.886105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.886105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:20:58.886105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.886105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:58.886105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:58.886105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:58.886105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.886813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:20:58.886813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:20:58.886813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:58.886813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:58.886813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:58.886813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:58.886813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:58.886813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:58.886813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:58.886813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:58.886813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:58.886813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:58.886813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:58.887483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:58.887483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:58.887483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:58.887483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:58.887483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:58.887483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:58.887483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:58.887483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:58.887483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:58.887483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.887483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:58.887483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.888194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:58.888194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:58.888194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:58.888194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:58.888194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:58.888194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.888194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:58.888194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:58.888194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.888194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:58.888194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:58.888194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.888194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:58.888194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:58.888194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.888194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:20:58.888194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:20:58.888194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:20:58.888194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:20:58.888736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:58.888736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:58.888736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:58.888736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:58.888736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:58.888736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:58.888736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:58.888736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:58.888736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:58.888736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:58.888736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:58.888736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:58.888736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:58.888736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:58.888736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:58.888736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:58.888736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:58.888736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:58.888736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.888736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:20:58.889270 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:20:58.889270 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:20:58.889270 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:20:58.889270 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:58.889270 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:20:58.889270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.889466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:20:58.889466 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:58.889466 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:20:58.889466 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:20:58.889466 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:58.889466 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:20:58.889466 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.889466 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:20:58.890278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.890278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:20:58.890278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:58.890278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:20:58.890278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.890278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:20:58.890278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:58.890278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:58.890278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:58.890278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:58.890278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:58.890278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:58.890659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:20:58.890659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:58.890659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:20:58.890659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:58.890659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:20:58.890659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:20:58.890659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:58.890659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:20:58.890659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:58.890659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:20:58.890659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:20:58.890659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:58.890659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:20:58.891048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.891048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:20:58.891306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.891306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:20:58.891306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:20:58.891306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:20:58.891306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:20:58.891306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:20:58.891306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.891306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:20:58.891306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:20:58.891306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.891306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:20:58.891306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:20:58.891306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.891306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:20:58.891306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:20:58.891306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:20:58.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:20:58.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:20:58.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:58.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:58.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:20:58.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:20:58.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:20:58.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:20:58.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:20:58.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:20:58.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:20:58.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:20:58.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:20:58.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:20:58.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:20:58.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:20:58.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:20:58.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:20:58.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:20:58.892369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:20:58.892369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:20:58.892369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:03.053607 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:03.054314 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:03.054314 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:03.055394 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:03.056960 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=236 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:21:03.085505 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:03.085819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.086584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:03.086584 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:03.086584 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:03.086584 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:03.086584 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:03.086584 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:03.086584 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.086584 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:03.087602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.087602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:03.087602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:03.087602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:03.087602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.087602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:21:03.087602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:03.087602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:03.087602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:03.087602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:03.087602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:03.087602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:03.088402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:03.088402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:03.088402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:03.088402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:03.088402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:03.088402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:03.088402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:03.088402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:03.088402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:03.088402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:03.088402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:03.088402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:03.088402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:03.089105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.089105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:03.089105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.089105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:03.089105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:03.089105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:03.089105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:03.089105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:03.089105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.089105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:03.089105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:03.089105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.089105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:03.089105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:03.089105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.089832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:03.089832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:03.089832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.089832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:21:03.089832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:03.089832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:03.089832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:03.089832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:03.089832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:03.089832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:03.089832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:03.089832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:03.089832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:03.089832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:03.089832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:03.089832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:03.089832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:03.089832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:03.089832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:03.091286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:03.091286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:03.091286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:03.091286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:03.091286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.091286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:03.793638 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:03.794059 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:03.794059 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:03.795124 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:03.795468 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:03.795731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.796774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:03.796774 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:03.796774 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:03.796774 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:03.796774 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:03.796774 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:03.796774 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.796774 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:03.798391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.798391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:03.798391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:03.798391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:03.798391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.798391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:21:03.798391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:03.798391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:03.798391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:03.798391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:03.798391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:03.798391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:03.799112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:03.799112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:03.799112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:03.799112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:03.799112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:03.799112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:03.799112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:03.799112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:03.799112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:03.799112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:03.799112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:03.799112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:03.799112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:03.799819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.799819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:03.800406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.800406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:03.800406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:03.800406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:03.800406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:03.800406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:03.800406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.800406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:03.800406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:03.800406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.800406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:03.800406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:03.800406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.800406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:03.800406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:03.800406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.800959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:21:03.800959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:03.800959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:03.800959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:03.800959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:03.800959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:03.800959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:03.800959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:03.800959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:03.800959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:03.800959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:03.800959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:03.800959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:03.800959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:03.800959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:03.800959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:03.800959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:03.800959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:03.800959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:03.801506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:03.801506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.801506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:03.836723 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:03.837016 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:03.837129 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:03.837256 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:21:03.838033 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:03.838471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.839345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:03.839345 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:03.839345 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:03.839345 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:03.839345 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:03.839345 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:03.839345 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.839345 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:03.840248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.840248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:21:03.840248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:03.840248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:03.840248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:03.840248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:03.840248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:03.840248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:03.840248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:03.840248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:03.840248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:03.840248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:21:03.840248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:03.840248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:21:03.840248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:21:03.840864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:03.840864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:21:03.840864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:03.840864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:21:03.840864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:21:03.840864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:03.840864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:21:03.840864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.840864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:21:03.841149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.841149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:03.841149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:03.841149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:03.841149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.841149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:21:03.841149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:21:03.841149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:21:03.841149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:03.841149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:03.841149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:03.841149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:03.841499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:03.841499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:03.841499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:03.841499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:03.841499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:03.841499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:03.841499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:03.841499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:03.841499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:03.841499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:03.841499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:03.841499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:03.841499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:03.841499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:03.842005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:03.842005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:03.842005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.842005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:03.842183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.842183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:03.842183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:03.842183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:03.842183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:03.842183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:03.842183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.842183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:03.842183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:03.842183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.842183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:03.842183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:03.842183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.842183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:03.842183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:03.842183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.842655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:21:03.842655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:21:03.842655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:21:03.842655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:03.842655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:03.842655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:03.842655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:03.842655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:03.842655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:03.842655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:03.842655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:03.842655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:03.842655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:03.842655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:03.842655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:03.842655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:03.842655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:03.842655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:03.843193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:03.843193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:03.843193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:03.843193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:03.843193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:03.843193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.843193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:03.876761 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:03.877051 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:03.877172 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:03.877328 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:03.877971 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:03.878328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.879120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:03.879120 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:03.879120 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:03.879120 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:03.879120 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:03.879120 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:03.879120 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.879120 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:03.880253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.880253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:21:03.880253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:03.880253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:03.880253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:03.880253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:03.880253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:03.880253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:03.880253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:03.880253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:03.880253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:03.880253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:21:03.880253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:03.880253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:21:03.880253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:21:03.880894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:03.880894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:21:03.880894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:03.880894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:21:03.880894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:21:03.880894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:03.880894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:21:03.880894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.880894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:21:03.881401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.881401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:03.881401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:03.881401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:03.881401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.881401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:21:03.881401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:21:03.881401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:03.881401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:03.881401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:03.881401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:03.881879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:03.881879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:03.881879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:03.881879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:03.881879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:03.881879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:03.881879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:03.881879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:03.881879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:03.881879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:03.881879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:03.881879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:03.881879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:03.881879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:03.882447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:03.882447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:03.882447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.882447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:03.882888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.882888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:03.882888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:03.882888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:03.882888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:03.882888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:03.882888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.882888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:03.882888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:03.882888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.882888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:03.882888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:03.882888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.882888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:03.882888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:03.882888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.882888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:21:03.883619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:21:03.883619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:03.883619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:03.883619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:03.883619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:03.883619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:03.883619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:03.883619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:03.883619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:03.883619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:03.883619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:03.883619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:03.883619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:03.883619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:03.883619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:03.883619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:03.883619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:03.883619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:03.883619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:03.885633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:03.885633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:03.885633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.885633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:03.886357 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:03.886519 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:03.886883 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:03.886980 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:03.887584 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:03.888014 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:03.888340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.889126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:03.889126 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:03.889126 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:03.889126 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:03.889126 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:03.889126 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:03.889126 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.889126 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:03.889972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.889972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:03.889972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:03.889972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:03.889972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.889972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:21:03.889972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:03.889972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:03.889972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:03.889972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:03.889972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:03.889972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:03.890498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:03.890498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:03.890498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:03.890498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:03.890498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:03.890498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:03.890498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:03.890498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:03.890498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:03.890498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:03.890498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:03.890498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:03.890498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:03.890900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.890900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:03.890965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.890965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:03.890965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:03.890965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:03.890965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:03.890965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:03.890965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.890965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:03.890965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:03.890965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.890965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:03.890965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:03.890965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.890965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:03.890965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:03.890965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:03.891694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:21:03.891694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:03.891694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:03.891694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:03.891694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:03.891694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:03.891694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:03.891694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:03.891694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:03.891694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:03.891694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:03.891694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:03.891694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:03.891694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:03.891694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:03.891694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:03.891694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:03.891694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:03.891694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:03.893082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:03.893082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:03.893082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:08.054084 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:08.055071 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:08.055071 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:08.056119 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:08.090667 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:08.090908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.091505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:08.091505 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:08.091505 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:08.091505 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:08.091505 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:08.091505 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:08.091505 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.091505 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:08.092517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.092517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:08.092517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:08.092517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:08.092517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.092517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:21:08.092517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:08.092517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:08.092517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:08.092517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:08.092517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:08.092517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:08.093183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:08.093183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:08.093183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:08.093183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:08.093183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:08.093183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:08.093183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:08.093183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:08.093183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:08.093183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:08.093183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:08.093183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:08.093183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:08.093831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.093831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:08.093831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.093831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:08.093831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:08.093831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:08.093831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:08.093831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:08.093831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.093831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:08.093831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:08.093831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.093831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:08.093831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:08.093831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.094504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:08.094504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:08.094504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.094504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:21:08.094504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:08.094504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:08.094504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:08.094504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:08.094504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:08.094504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:08.094504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:08.094504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:08.094504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:08.094504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:08.094504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:08.094504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:08.094504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:08.094504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:08.094504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:08.095362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:08.095362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:08.095362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:08.095362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:08.095362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.095362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:08.794222 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:08.794786 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:08.794991 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:08.796478 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:08.797009 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:08.797272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.798175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:08.798175 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:08.798175 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:08.798175 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:08.798175 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:08.798175 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:08.798175 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.798175 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:08.799165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.799165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:08.799165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:08.799165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:08.799165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.799165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:21:08.799165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:08.799165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:08.799165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:08.799165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:08.799165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:08.799165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:08.799790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:08.799790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:08.799790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:08.799790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:08.799790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:08.799790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:08.799790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:08.799790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:08.799790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:08.799790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:08.799790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:08.799790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:08.799790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:08.800293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.800293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:08.800293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.800293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:08.800293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:08.800293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:08.800293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:08.800293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:08.800293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.800293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:08.800293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:08.800293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.800293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:08.800293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:08.800293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.801275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:08.801275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:08.801275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.801275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:21:08.801275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:08.801275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:08.801275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:08.801275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:08.801275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:08.801275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:08.801275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:08.801275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:08.801275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:08.801275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:08.801275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:08.801275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:08.801275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:08.801275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:08.801275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:08.802174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:08.802174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:08.802174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:08.802174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:08.802174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.802174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:08.837818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:08.837818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:08.838034 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:08.838070 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:21:08.838902 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:08.839246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.840165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:08.840165 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:08.840165 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:08.840165 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:08.840165 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:08.840165 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:08.840165 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.840165 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:08.841516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.841516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:21:08.841516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:08.841516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:08.841516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:08.841516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:08.841516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:08.841516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:08.841516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:08.841516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:08.841516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:08.841516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:21:08.841516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:08.841516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:21:08.841516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:21:08.841930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:08.841930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:21:08.841930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:08.841930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:21:08.841930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:21:08.841930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:08.841930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:21:08.841930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.841930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:21:08.843146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.843146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:08.843146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:08.843146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:08.843146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.843146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:21:08.843146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:21:08.843146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:21:08.843146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:08.843146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:08.843146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:08.843146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:08.843509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:08.843509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:08.843509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:08.843509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:08.843509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:08.843509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:08.843509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:08.843509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:08.843509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:08.843509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:08.843509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:08.843509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:08.843509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:08.843509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:08.843843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:08.843843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:08.843843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.843843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:08.844953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.844953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:08.844953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:08.844953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:08.844953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:08.844953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:08.844953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.844953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:08.844953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:08.844953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.844953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:08.844953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:08.844953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.844953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:08.844953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:08.844953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.845396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:21:08.845396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:21:08.845396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:21:08.845396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:08.845396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:08.845396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:08.845396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:08.845396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:08.845396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:08.845396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:08.845396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:08.845396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:08.845396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:08.845396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:08.845396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:08.845396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:08.845396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:08.845396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:08.845950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:08.845950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:08.845950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:08.845950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:08.845950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:08.845950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.845950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:08.877192 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:08.877518 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:08.877790 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:08.877918 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:08.878633 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:08.879201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.880220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:08.880220 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:08.880220 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:08.880220 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:08.880220 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:08.880220 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:08.880220 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.880220 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:08.881628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.881628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:21:08.881628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:08.881628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:08.881628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:08.881628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:08.881628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:08.881628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:08.881628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:08.881628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:08.881628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:08.881628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:21:08.881628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:08.881628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:21:08.881628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:21:08.882533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:08.882533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:21:08.882533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:08.882533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:21:08.882533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:21:08.882533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:08.882533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:21:08.882533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.882533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:21:08.883209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.883209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:08.883209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:08.883209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:08.883209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.883209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:21:08.883209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:21:08.883209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:08.883209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:08.883209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:08.883209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:08.884046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:08.884046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:08.884046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:08.884046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:08.884046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:08.884046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:08.884046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:08.884046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:08.884046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:08.884046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:08.884046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:08.884046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:08.884046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:08.884046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:08.885043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:08.885043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:08.885043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.885043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:08.885043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.885043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:08.885043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:08.885043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:08.885043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:08.885043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:08.885043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.885043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:08.885043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:08.885043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.886005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:08.886005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:08.886005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.886005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:08.886005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:08.886005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.886005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:21:08.886005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:21:08.886005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:08.886005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:08.886005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:08.886005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:08.886005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:08.886005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:08.886005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:08.886005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:08.886005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:08.886005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:08.886005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:08.890857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:08.890857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:08.890857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:08.890857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:08.890857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:08.890857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:08.890857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:08.890857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:08.890857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:08.890857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.890857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:08.890857 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:08.890857 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:08.891844 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:08.891844 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:08.891844 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:08.891844 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:08.891844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.891844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:08.891844 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:08.891844 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:08.891844 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:08.892202 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:08.892202 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:08.892202 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.892202 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:08.892202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.892202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:08.892202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:08.892202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:08.892202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.892202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:21:08.892202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:08.892202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:08.892202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:08.892202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:08.892779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:08.892779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:08.892779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:08.892779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:08.892779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:08.892779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:08.892779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:08.892779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:08.892779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:08.892779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:08.892779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:08.892779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:08.892779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:08.892779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:08.893300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:08.893300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.893300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:08.893300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.893300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:08.893300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:08.893300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:08.893300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:08.893300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:08.893300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.893300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:08.893300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:08.893300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.893300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:08.893300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:08.893300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.893916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:08.893916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:08.893916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:08.893916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:21:08.893916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:08.893916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:08.893916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:08.893916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:08.893916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:08.893916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:08.893916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:08.893916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:08.893916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:08.893916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:08.893916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:08.893916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:08.893916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:08.893916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:08.893916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:08.894765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:08.894765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:08.894765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:08.894765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:08.894765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:08.894765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:13.054150 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:13.054673 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:13.054673 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:13.054891 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:13.065088 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
Apr 28 20:21:13.065249 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, security_groups=2, routers=1, router_ports=1 {{(pid=106506) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
Apr 28 20:21:13.065378 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 7ff08466-af6d-4e13-bb1d-8987170a5734 (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:21:13.100729 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-7ff08466-af6d-4e13-bb1d-8987170a5734, resource={'id': '7ff08466-af6d-4e13-bb1d-8987170a5734', 'name': 'private', 'tenant_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'project_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['4b461bd9-09ea-4f87-a007-a11ab022ff7f', '8f3677a2-e2be-443a-a90c-2cd7b3095c49'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-28T20:10:58Z', 'updated_at': '2026-04-28T20:11:02Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 46342}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.100907 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-7ff08466-af6d-4e13-bb1d-8987170a5734, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.101339 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-27fbc209-4b05-4b4a-8d5c-40d6d87b7e22, external_ids_update=None, columns={'tag': 46342}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.102701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.102701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:13.102701 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.102701 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:13.102701 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.102701 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.102701 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:13.102701 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.102701 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:13.103226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 7ff08466-af6d-4e13-bb1d-8987170a5734 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.103226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:13.103226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:21:13.103226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:21:13.103226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:21:13.103226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:21:13.103226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:21:13.103226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:13.103226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.103226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:13.103226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:13.103469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:13.103469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.103469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:13.103469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:13.103469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:13.103469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:13.103469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:13.103469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:13.103469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:13.103469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:13.103469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.103469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:13.103469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.103469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.103756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:13.103756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.103756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:13.103756 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 8a1fb774-5693-4139-8b75-6ab5273228c7 (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:21:13.138764 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, resource={'id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'name': 'public', 'tenant_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'project_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['335d12ce-2d0c-4086-b3f4-2af390c03d70', '5308c10f-28d1-441c-acb2-d1d6dc3d2ece'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-04-28T20:11:05Z', 'updated_at': '2026-04-28T20:11:13Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.139007 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.139362 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-7dbf713e-d963-4884-ae64-da4ce196e719, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.139586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.140378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:13.140378 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.140378 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:13.140378 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.140378 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.140378 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:13.140378 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.140378 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:13.142644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 8a1fb774-5693-4139-8b75-6ab5273228c7 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.142644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:13.142644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:21:13.142644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:21:13.142644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:21:13.142644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:21:13.142644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:21:13.142644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:13.142644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.142644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:13.142644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:13.142930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:13.142930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.142930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:13.142930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:13.142930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:13.142930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:13.142930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:13.142930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:13.142930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:13.142930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:13.142930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.142930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:13.142930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.142930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.143212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:13.143212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.143212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:13.143212 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource c6718637-daa9-4db1-bd11-63abe3891db6 (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:21:13.174882 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-c6718637-daa9-4db1-bd11-63abe3891db6, resource={'id': 'c6718637-daa9-4db1-bd11-63abe3891db6', 'name': 'admin_net', 'tenant_id': '3eb96b75a3044dd0a338616e1d9081d0', 'project_id': '3eb96b75a3044dd0a338616e1d9081d0', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['c1cc9883-a7e2-4abc-8892-a7c1b200d3ad'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-28T20:15:13Z', 'updated_at': '2026-04-28T20:15:14Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 7626}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.175141 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-c6718637-daa9-4db1-bd11-63abe3891db6, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.175498 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-7461d8df-a8a0-40c3-87a1-26ed88f9f7ce, external_ids_update=None, columns={'tag': 7626}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.175622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.176286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:13.176286 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.176286 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:13.176286 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.176286 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.176286 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:13.176286 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.176286 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:13.177432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource c6718637-daa9-4db1-bd11-63abe3891db6 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.177432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:13.177432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:21:13.177432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:21:13.177432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:21:13.177432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:21:13.177432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:21:13.177432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:13.177432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.177432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:13.177432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:13.177718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:13.177718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.177718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:13.177718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:13.177718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:13.177718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:13.177718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:13.177718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:13.177718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:13.177718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:13.177718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.177718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:13.177718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.177718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.177981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:13.177981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.177981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:13.177981 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource d3dfebbd-7943-4813-b509-739084ada63b (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:21:13.206800 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d3dfebbd-7943-4813-b509-739084ada63b, resource={'id': 'd3dfebbd-7943-4813-b509-739084ada63b', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-pi48i', 'tenant_id': 'a11fcd8844504e7eaf509ece683e8c8f', 'project_id': 'a11fcd8844504e7eaf509ece683e8c8f', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['c68bc19b-c88a-4cd2-bf3a-d847d267d23f'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-28T20:15:49Z', 'updated_at': '2026-04-28T20:15:50Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 58143}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.207055 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-d3dfebbd-7943-4813-b509-739084ada63b, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-pi48i', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.207230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-ed33b0a1-b0b7-4e33-aa96-92c114a82f5d, external_ids_update=None, columns={'tag': 58143}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.207316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.207994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:13.207994 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.207994 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:13.207994 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.207994 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.207994 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:13.207994 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.207994 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:13.209350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource d3dfebbd-7943-4813-b509-739084ada63b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.209350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:13.209350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:21:13.209350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:21:13.209350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:21:13.209350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:21:13.209350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:21:13.209350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:13.209350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.209350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:13.209350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:13.209600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:13.209600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.209600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:13.209600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:13.209600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:13.209600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:13.209600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:13.209600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:13.209600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:13.209600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:13.209600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.209600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:13.209600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.209600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.209892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:13.209892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.209892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:13.209892 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource eba8d7be-8417-46d9-a325-7fe5863e6557 (type: security_groups) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:21:13.253000 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.db.ovn_revision_numbers_db [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Successfully bumped revision number for resource eba8d7be-8417-46d9-a325-7fe5863e6557 (type: security_groups) to 10
Apr 28 20:21:13.253200 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 37cff69a-12a0-460f-9648-2dedf087d771 (type: security_groups) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:21:13.299428 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.db.ovn_revision_numbers_db [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Successfully bumped revision number for resource 37cff69a-12a0-460f-9648-2dedf087d771 (type: security_groups) to 15
Apr 28 20:21:13.299641 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 6feb526a-13c6-41ac-983d-ea1d2251ff3d (type: routers) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:21:13.404499 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, resource={'id': '6feb526a-13c6-41ac-983d-ea1d2251ff3d', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-pi48i', 'project_id': 'a11fcd8844504e7eaf509ece683e8c8f', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'external_fixed_ips': [{'subnet_id': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece', 'ip_address': '172.24.5.87'}, {'subnet_id': '335d12ce-2d0c-4086-b3f4-2af390c03d70', 'ip_address': '2001:db8::2d8'}], 'enable_snat': True}, 'gw_port_id': '73493630-b0ac-48ec-be98-5b048d2e2ce7', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-pi48i', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'external_fixed_ips': [{'ip_address': '172.24.5.87', 'subnet_id': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece'}, {'ip_address': '2001:db8::2d8', 'subnet_id': '335d12ce-2d0c-4086-b3f4-2af390c03d70'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-04-28T20:15:51Z', 'updated_at': '2026-04-28T20:15:54Z', 'revision_number': 3, 'tenant_id': 'a11fcd8844504e7eaf509ece683e8c8f'}, resource_type=routers, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.404847 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-pi48i', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.405225 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "73493630-b0ac-48ec-be98-5b048d2e2ce7"')], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.405505 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "73493630-b0ac-48ec-be98-5b048d2e2ce7"')], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.405776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.406636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:13.406636 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.406636 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:13.406636 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.406636 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.406636 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:13.406636 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.406636 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:13.406881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Unable to update router 6feb526a-13c6-41ac-983d-ea1d2251ff3d. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.408754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 6feb526a-13c6-41ac-983d-ea1d2251ff3d (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.408754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:13.408754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:21:13.408754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:21:13.408754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:21:13.408754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:21:13.408754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
Apr 28 20:21:13.408754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Apr 28 20:21:13.408754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
Apr 28 20:21:13.408754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Apr 28 20:21:13.408754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Apr 28 20:21:13.409041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Apr 28 20:21:13.409041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
Apr 28 20:21:13.409041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:13.409041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.409041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:13.409041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:13.409041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:13.409041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.409041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:13.409041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:13.409041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:13.409041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:13.409041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:13.409041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:13.409311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:13.409311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:13.409311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.409311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:13.409311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.409311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.409311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:13.409311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.409311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:13.409311 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 73493630-b0ac-48ec-be98-5b048d2e2ce7 (type: router_ports) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:21:13.554145 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-73493630-b0ac-48ec-be98-5b048d2e2ce7, resource={'id': '73493630-b0ac-48ec-be98-5b048d2e2ce7', 'name': '', 'network_id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'project_id': '', 'mac_address': 'fa:16:3e:f8:07:38', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6feb526a-13c6-41ac-983d-ea1d2251ff3d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece', 'ip_address': '172.24.5.87'}, {'subnet_id': '335d12ce-2d0c-4086-b3f4-2af390c03d70', 'ip_address': '2001:db8::2d8'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000171185', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-04-28T20:15:51Z', 'updated_at': '2026-04-28T20:15:54Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.554765 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-73493630-b0ac-48ec-be98-5b048d2e2ce7, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece 335d12ce-2d0c-4086-b3f4-2af390c03d70', 'neutron:network_name': 'neutron-8a1fb774-5693-4139-8b75-6ab5273228c7', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.87/24', '2001:db8::2d8/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.555469 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=73493630-b0ac-48ec-be98-5b048d2e2ce7, lrouter_port=lrp-73493630-b0ac-48ec-be98-5b048d2e2ce7, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.556000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.557585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:13.557585 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.557585 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:13.557585 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.557585 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.557585 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:13.557585 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.557585 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:13.559130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 73493630-b0ac-48ec-be98-5b048d2e2ce7 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.559130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:13.559130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:21:13.559130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:21:13.559130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:21:13.559130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:21:13.559130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1971, in update_router_port
Apr 28 20:21:13.559130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:13.559130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.559130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:13.559130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:13.559383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:13.559383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.559383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:13.559383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:13.559383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:13.559383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:13.559383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:13.559383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:13.559383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:13.559383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:13.559383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.559383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:13.559383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.559383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.559645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:13.559645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.559645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:13.559645 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_for_inconsistencies finished in 0.504 seconds
Apr 28 20:21:13.559645 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:13.559645 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:13.560428 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:13.596901 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.597347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.598132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:13.598132 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.598132 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:13.598132 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.598132 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.598132 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:13.598132 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.598132 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:13.598984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.598984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:13.598984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:13.598984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:13.598984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.598984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:21:13.598984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:13.598984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.598984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:13.598984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:13.598984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:13.598984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.599286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:13.599286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:13.599286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:13.599286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:13.599286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:13.599286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:13.599286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:13.599286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:13.599286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.599286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:13.599286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.599286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.599286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:13.599529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.599529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:13.600009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.600009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:13.600009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:13.600009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:13.600009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:13.600009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:13.600009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.600009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:13.600009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:13.600009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.600009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:13.600009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:13.600009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.600009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:13.600009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:13.600009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.600333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:21:13.600333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:13.600333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.600333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:13.600333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:13.600333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:13.600333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.600333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:13.600333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:13.600333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:13.600333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:13.600333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:13.600333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:13.600333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:13.600333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:13.600333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.600333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:13.600333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.600333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.600690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:13.600690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.600690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:13.794860 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:13.795452 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:13.795656 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:13.797561 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.797855 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.798082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.798936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:13.798936 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.798936 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:13.798936 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.798936 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.798936 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:13.798936 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.798936 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:13.800877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.800877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:13.800877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:13.800877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:13.800877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.800877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:21:13.800877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:13.800877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.800877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:13.800877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:13.800877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:13.800877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.801184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:13.801184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:13.801184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:13.801184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:13.801184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:13.801184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:13.801184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:13.801184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:13.801184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.801184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:13.801184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.801184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.801184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:13.801433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.801433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:13.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:13.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:13.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:13.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:13.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:13.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:13.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:13.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:13.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:13.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:13.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:13.802908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.803250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:21:13.803250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:13.803250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.803250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:13.803250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:13.803250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:13.803250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.803250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:13.803250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:13.803250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:13.803250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:13.803250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:13.803250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:13.803250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:13.803250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:13.803250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.803250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:13.803250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.803250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.803590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:13.803590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.803590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:13.837965 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:13.838250 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:13.838369 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:13.838493 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:21:13.838972 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.839193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.839778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:13.839778 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.839778 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:13.839778 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.839778 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.839778 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:13.839778 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.839778 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:13.840587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.840587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:21:13.840587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:13.840587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:13.840587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.840587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:13.840587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:13.840587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:13.840587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.840587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:13.840587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:13.840587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:21:13.840587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:13.840587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:21:13.840587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:21:13.840953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:13.840953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:21:13.840953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.840953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:21:13.840953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.840953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.840953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:21:13.840953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.840953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:21:13.841535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.841535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:13.841535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:13.841535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:13.841535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.841535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:21:13.841535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:21:13.841535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:21:13.841535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:13.841535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:13.841535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.841535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:13.841825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:13.841825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:13.841825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.841825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:13.841825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:13.841825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:13.841825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:13.841825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:13.841825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:13.841825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:13.841825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:13.841825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.841825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:13.841825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.842094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.842094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:13.842094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.842094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:13.843704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.843704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:13.843704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:13.843704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:13.843704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:13.843704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:13.843704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.843704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:13.843704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:13.843704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.843704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:13.843704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:13.843704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.843704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:13.843704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:13.843704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.844037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:21:13.844037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:21:13.844037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:21:13.844037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:13.844037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:13.844037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.844037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:13.844037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:13.844037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:13.844037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.844037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:13.844037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:13.844037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:13.844037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:13.844037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:13.844037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:13.844037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:13.844037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:13.844382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.844382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:13.844382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.844382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.844382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:13.844382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.844382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:13.877823 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:13.878219 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:13.878422 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:13.878659 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:13.879439 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.879847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.880667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:13.880667 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.880667 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:13.880667 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.880667 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.880667 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:13.880667 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.880667 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:13.882166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.882166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:21:13.882166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:13.882166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:13.882166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.882166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:13.882166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:13.882166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:13.882166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.882166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:13.882166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:13.882166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:21:13.882166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:13.882166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:21:13.882166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:21:13.882506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:13.882506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:21:13.882506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.882506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:21:13.882506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.882506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.882506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:21:13.882506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.882506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:21:13.883451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.883451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:13.883451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:13.883451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:13.883451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.883451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:21:13.883451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:21:13.883451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:13.883451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:13.883451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.883451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:13.883783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:13.883783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:13.883783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.883783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:13.883783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:13.883783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:13.883783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:13.883783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:13.883783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:13.883783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:13.883783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:13.883783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.883783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:13.883783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.884056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.884056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:13.884056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.884056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:13.886013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.886013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:13.886013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:13.886013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:13.886013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:13.886013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:13.886013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.886013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:13.886013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:13.886013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.886013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:13.886013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:13.886013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.886013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:13.886013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:13.886013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.886013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:21:13.886382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:21:13.886382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:13.886382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:13.886382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.886382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:13.886382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:13.886382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:13.886382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.886382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:13.886382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:13.886382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:13.886382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:13.886382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:13.886382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:13.886382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:13.886382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:13.886382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.886382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:13.886382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.886757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.886757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:13.886757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.886757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:13.887199 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:13.887739 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:13.887976 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:13.888208 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:13.888580 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.888866 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:13.888997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.889408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:13.889408 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.889408 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:13.889408 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.889408 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.889408 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:13.889408 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.889408 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:13.890018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.890018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:13.890018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:13.890018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:13.890018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.890018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:21:13.890018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:13.890018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.890018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:13.890018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:13.890018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:13.890018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.890309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:13.890309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:13.890309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:13.890309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:13.890309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:13.890309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:13.890309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:13.890309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:13.890309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.890309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:13.890309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.890309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.890309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:13.890568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.890568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:13.892701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.892701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:13.892701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:13.892701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:13.892701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:13.892701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:13.892701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.892701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:13.892701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:13.892701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.892701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:13.892701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:13.892701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.892701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:13.892701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:13.892701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:13.893054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:21:13.893054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:13.893054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.893054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:13.893054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:13.893054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:13.893054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:13.893054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:13.893054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:13.893054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:13.893054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:13.893054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:13.893054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:13.893054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:13.893054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:13.893054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:13.893054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:13.893054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:13.893054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:13.893412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:13.893412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:13.893412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:18.559846 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:18.560982 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:18.561119 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:18.561911 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:18.589952 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:18.590318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.591217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:18.591217 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:18.591217 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:18.591217 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:18.591217 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:18.591217 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:18.591217 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.591217 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:18.592130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.592130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:18.592130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:18.592130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:18.592130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.592130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:21:18.592130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:18.592130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:18.592130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:18.592130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:18.592130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:18.592130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:18.593274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:18.593274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:18.593274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:18.593274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:18.593274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:18.593274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:18.593274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:18.593274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:18.593274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:18.593274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:18.593274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:18.593274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:18.593274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:18.594042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.594042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:18.594042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.594042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:18.594042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:18.594042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:18.594042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:18.594042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:18.594042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.594042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:18.594042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:18.594042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.594042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:18.594042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:18.594042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.594838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:18.594838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:18.594838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.594838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:21:18.594838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:18.594838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:18.594838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:18.594838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:18.594838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:18.594838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:18.594838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:18.594838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:18.594838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:18.594838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:18.594838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:18.594838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:18.594838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:18.594838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:18.594838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:18.597123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:18.597123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:18.597123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:18.597123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:18.597123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.597123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:18.795924 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:18.796398 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:18.796598 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:18.797566 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:18.797866 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:18.798002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.798603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:18.798603 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:18.798603 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:18.798603 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:18.798603 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:18.798603 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:18.798603 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.798603 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:18.799758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.799758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:18.799758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:18.799758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:18.799758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.799758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:21:18.799758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:18.799758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:18.799758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:18.799758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:18.799758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:18.799758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:18.800234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:18.800234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:18.800234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:18.800234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:18.800234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:18.800234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:18.800234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:18.800234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:18.800234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:18.800234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:18.800234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:18.800234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:18.800234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:18.800707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.800707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:18.800916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.800916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:18.800916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:18.800916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:18.800916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:18.800916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:18.800916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.800916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:18.800916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:18.800916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.800916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:18.800916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:18.800916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.800916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:18.800916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:18.800916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.801440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:21:18.801440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:18.801440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:18.801440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:18.801440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:18.801440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:18.801440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:18.801440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:18.801440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:18.801440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:18.801440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:18.801440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:18.801440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:18.801440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:18.801440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:18.801440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:18.801440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:18.801440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:18.801440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:18.802069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:18.802069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.802069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:18.838391 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:18.838664 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:18.838855 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:18.839041 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:21:18.839653 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:18.839911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.840770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:18.840770 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:18.840770 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:18.840770 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:18.840770 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:18.840770 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:18.840770 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.840770 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:18.841821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.841821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:21:18.841821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:18.841821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:18.841821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:18.841821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:18.841821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:18.841821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:18.841821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:18.841821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:18.841821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:18.841821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:21:18.841821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:18.841821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:21:18.841821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:21:18.842399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:18.842399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:21:18.842399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:18.842399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:21:18.842399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:21:18.842399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:18.842399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:21:18.842399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.842399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:21:18.843121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.843121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:18.843121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:18.843121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:18.843121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.843121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:21:18.843121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:21:18.843121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:21:18.843121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:18.843121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:18.843121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:18.843121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:18.843519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:18.843519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:18.843519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:18.843519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:18.843519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:18.843519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:18.843519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:18.843519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:18.843519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:18.843519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:18.843519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:18.843519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:18.843519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:18.843519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:18.844290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:18.844290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:18.844290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.844290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:18.844690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.844690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:18.844690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:18.844690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:18.844690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:18.844690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:18.844690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.844690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:18.844690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:18.844690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.844690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:18.844690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:18.844690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.844690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:18.844690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:18.844690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.845824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:21:18.845824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:21:18.845824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:21:18.845824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:18.845824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:18.845824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:18.845824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:18.845824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:18.845824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:18.845824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:18.845824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:18.845824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:18.845824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:18.845824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:18.845824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:18.845824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:18.845824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:18.845824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:18.846337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:18.846337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:18.846337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:18.846337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:18.846337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:18.846337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.846337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:18.878324 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:18.878640 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:18.878822 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:18.879006 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:18.879814 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:18.880339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.881510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:18.881510 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:18.881510 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:18.881510 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:18.881510 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:18.881510 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:18.881510 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.881510 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:18.882636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.882636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:21:18.882636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:18.882636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:18.882636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:18.882636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:18.882636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:18.882636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:18.882636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:18.882636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:18.882636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:18.882636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:21:18.882636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:18.882636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:21:18.882636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:21:18.883289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:18.883289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:21:18.883289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:18.883289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:21:18.883289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:21:18.883289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:18.883289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:21:18.883289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.883289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:21:18.884022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.884022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:18.884022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:18.884022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:18.884022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.884022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:21:18.884022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:21:18.884022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:18.884022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:18.884022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:18.884022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:18.884535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:18.884535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:18.884535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:18.884535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:18.884535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:18.884535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:18.884535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:18.884535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:18.884535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:18.884535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:18.884535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:18.884535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:18.884535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:18.884535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:18.885141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:18.885141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:18.885141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.885141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:18.885691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.885691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:18.885691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:18.885691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:18.885691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:18.885691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:18.885691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.885691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:18.885691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:18.885691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.885691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:18.885691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:18.885691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.885691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:18.885691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:18.885691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.885691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:21:18.886480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:21:18.886480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:18.886480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:18.886480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:18.886480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:18.886480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:18.886480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:18.886480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:18.886480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:18.886480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:18.886480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:18.886480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:18.886480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:18.886480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:18.886480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:18.886480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:18.886480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:18.886480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:18.886480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:18.887284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:18.887284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:18.887284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.887284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:18.887737 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:18.887984 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:18.888141 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:18.888364 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:18.888852 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:18.889126 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:18.889281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.890245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:18.890245 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:18.890245 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:18.890245 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:18.890245 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:18.890245 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:18.890245 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.890245 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:18.891101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.891101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:18.891101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:18.891101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:18.891101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.891101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:21:18.891101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:18.891101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:18.891101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:18.891101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:18.891101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:18.891101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:18.891516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:18.891516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:18.891516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:18.891516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:18.891516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:18.891516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:18.891516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:18.891516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:18.891516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:18.891516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:18.891516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:18.891516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:18.891516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:18.892034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.892034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:18.892123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.892123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:18.892123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:18.892123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:18.892123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:18.892123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:18.892123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.892123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:18.892123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:18.892123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.892123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:18.892123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:18.892123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.892123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:18.892123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:18.892123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:18.892616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:21:18.892616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:18.892616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:18.892616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:18.892616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:18.892616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:18.892616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:18.892616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:18.892616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:18.892616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:18.892616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:18.892616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:18.892616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:18.892616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:18.892616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:18.892616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:18.892616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:18.892616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:18.892616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:18.893161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:18.893161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:18.893161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:23.561302 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:23.561945 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:23.561945 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:23.563494 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:23.599708 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:23.600255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.601066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:23.601066 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:23.601066 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:23.601066 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:23.601066 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:23.601066 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:23.601066 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.601066 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:23.601997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.601997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:23.601997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:23.601997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:23.601997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.601997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:21:23.601997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:23.601997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:23.601997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:23.601997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:23.601997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:23.601997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:23.602256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:23.602256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:23.602256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:23.602256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:23.602256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:23.602256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:23.602256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:23.602256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:23.602256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:23.602256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:23.602256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:23.602256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:23.602256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:23.602528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.602528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:23.603078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.603078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:23.603078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:23.603078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:23.603078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:23.603078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:23.603078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.603078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:23.603078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:23.603078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.603078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:23.603078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:23.603078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.603078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:23.603078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:23.603078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.603423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:21:23.603423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:23.603423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:23.603423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:23.603423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:23.603423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:23.603423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:23.603423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:23.603423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:23.603423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:23.603423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:23.603423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:23.603423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:23.603423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:23.603423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:23.603423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:23.603423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:23.603423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:23.603423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:23.603783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:23.603783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.603783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:23.796786 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:23.797415 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:23.797626 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:23.798618 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:23.798833 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:23.798962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.799499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:23.799499 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:23.799499 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:23.799499 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:23.799499 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:23.799499 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:23.799499 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.799499 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:23.800733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.800733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:23.800733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:23.800733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:23.800733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.800733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:21:23.800733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:23.800733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:23.800733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:23.800733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:23.800733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:23.800733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:23.800978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:23.800978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:23.800978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:23.800978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:23.800978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:23.800978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:23.800978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:23.800978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:23.800978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:23.800978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:23.800978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:23.800978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:23.800978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:23.801253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.801253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:23.801859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.801859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:23.801859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:23.801859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:23.801859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:23.801859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:23.801859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.801859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:23.801859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:23.801859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.801859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:23.801859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:23.801859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.801859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:23.801859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:23.801859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.802225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:21:23.802225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:23.802225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:23.802225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:23.802225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:23.802225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:23.802225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:23.802225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:23.802225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:23.802225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:23.802225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:23.802225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:23.802225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:23.802225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:23.802225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:23.802225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:23.802225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:23.802225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:23.802225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:23.802640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:23.802640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.802640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:23.858075 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:23.858075 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:23.858075 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:23.858075 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:21:23.858075 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:23.858075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.858374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:23.858374 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:23.858374 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:23.858374 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:23.858374 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:23.858374 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:23.858374 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.858374 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:23.858374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.858374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:21:23.858374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:23.858374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:23.858374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:23.858374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:23.858374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:23.858374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:23.858374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:23.858374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:23.858999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:23.858999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:21:23.858999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:23.858999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:21:23.858999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:21:23.858999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:23.858999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:21:23.858999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:23.858999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:21:23.858999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:21:23.858999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:23.858999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:21:23.858999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.858999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:21:23.858999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.859395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:23.859395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:23.859395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:23.859395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.859395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:21:23.859395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:21:23.859395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:21:23.859395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:23.859395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:23.859395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:23.859395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:23.859395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:23.859395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:23.859696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:23.859696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:23.859696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:23.859696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:23.859696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:23.859696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:23.859696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:23.859696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:23.859696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:23.859696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:23.859696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:23.859696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:23.859696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:23.859696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:23.860097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.860097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:23.860097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.860097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:23.860097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:23.860097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:23.860097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:23.860097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:23.860097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.860097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:23.860097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:23.860097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.860097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:23.860097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:23.860097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.860431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:23.860431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:23.860431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.860431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:21:23.860431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:21:23.860431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:21:23.860431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:23.860431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:23.860431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:23.860431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:23.860431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:23.860431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:23.860431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:23.860431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:23.860431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:23.860431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:23.860431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:23.860431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:23.860431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:23.860914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:23.860914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:23.860914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:23.860914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:23.860914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:23.860914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:23.860914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:23.860914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.860914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:23.879035 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:23.879388 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:23.879592 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:23.879894 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:23.880456 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:23.880718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.881311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:23.881311 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:23.881311 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:23.881311 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:23.881311 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:23.881311 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:23.881311 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.881311 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:23.882193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.882193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:21:23.882193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:23.882193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:23.882193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:23.882193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:23.882193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:23.882193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:23.882193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:23.882193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:23.882193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:23.882193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:21:23.882193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:23.882193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:21:23.882193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:21:23.882464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:23.882464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:21:23.882464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:23.882464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:21:23.882464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:21:23.882464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:23.882464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:21:23.882464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.882464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:21:23.883269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.883269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:23.883269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:23.883269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:23.883269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.883269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:21:23.883269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:21:23.883269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:23.883269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:23.883269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:23.883269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:23.883735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:23.883735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:23.883735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:23.883735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:23.883735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:23.883735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:23.883735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:23.883735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:23.883735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:23.883735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:23.883735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:23.883735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:23.883735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:23.883735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:23.884295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:23.884295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:23.884295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.884295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:23.884461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.884461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:23.884461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:23.884461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:23.884461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:23.884461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:23.884461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.884461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:23.884461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:23.884461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.884461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:23.884461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:23.884461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.884461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:23.884461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:23.884461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.884461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:21:23.885112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:21:23.885112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:23.885112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:23.885112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:23.885112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:23.885112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:23.885112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:23.885112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:23.885112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:23.885112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:23.885112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:23.885112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:23.885112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:23.885112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:23.885112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:23.885112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:23.885112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:23.885112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:23.885112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:23.885907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:23.885907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:23.885907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.885907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:23.888244 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:23.888485 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:23.888591 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:23.888823 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:23.889309 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:23.889561 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:23.889744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.890359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:23.890359 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:23.890359 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:23.890359 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:23.890359 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:23.890359 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:23.890359 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.890359 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:23.891418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.891418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:23.891418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:23.891418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:23.891418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.891418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:21:23.891418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:23.891418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:23.891418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:23.891418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:23.891418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:23.891418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:23.891754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:23.891754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:23.891754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:23.891754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:23.891754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:23.891754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:23.891754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:23.891754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:23.891754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:23.891754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:23.891754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:23.891754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:23.891754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:23.892386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.892386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:23.892692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.892692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:23.892692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:23.892692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:23.892692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:23.892692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:23.892692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.892692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:23.892692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:23.892692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.892692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:23.892692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:23.892692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.892692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:23.892692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:23.892692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:23.893130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:21:23.893130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:23.893130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:23.893130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:23.893130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:23.893130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:23.893130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:23.893130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:23.893130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:23.893130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:23.893130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:23.893130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:23.893130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:23.893130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:23.893130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:23.893130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:23.893130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:23.893130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:23.893130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:23.893618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:23.893618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:23.893618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:28.561911 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:28.562511 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:28.562511 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:28.563055 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:28.601414 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:28.601933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.603061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:28.603061 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:28.603061 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:28.603061 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:28.603061 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:28.603061 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:28.603061 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.603061 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:28.604179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.604179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:28.604179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:28.604179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:28.604179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.604179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:21:28.604179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:28.604179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:28.604179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:28.604179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:28.604179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:28.604179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:28.604839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:28.604839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:28.604839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:28.604839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:28.604839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:28.604839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:28.604839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:28.604839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:28.604839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:28.604839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:28.604839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:28.604839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:28.604839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:28.605511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.605511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:28.605989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.605989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:28.605989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:28.605989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:28.605989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:28.605989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:28.605989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.605989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:28.605989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:28.605989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.605989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:28.605989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:28.605989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.605989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:28.605989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:28.605989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.606858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:21:28.606858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:28.606858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:28.606858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:28.606858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:28.606858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:28.606858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:28.606858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:28.606858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:28.606858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:28.606858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:28.606858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:28.606858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:28.606858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:28.606858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:28.606858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:28.606858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:28.606858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:28.606858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:28.607847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:28.607847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.607847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:28.797384 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:28.797701 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:28.797899 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:28.798800 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:28.799018 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:28.799183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.800038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:28.800038 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:28.800038 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:28.800038 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:28.800038 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:28.800038 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:28.800038 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.800038 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:28.800912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.800912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:28.800912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:28.800912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:28.800912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.800912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:21:28.800912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:28.800912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:28.800912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:28.800912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:28.800912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:28.800912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:28.801272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:28.801272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:28.801272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:28.801272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:28.801272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:28.801272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:28.801272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:28.801272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:28.801272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:28.801272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:28.801272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:28.801272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:28.801272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:28.801510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.801510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:28.801652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.801652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:28.801652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:28.801652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:28.801652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:28.801652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:28.801652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.801652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:28.801652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:28.801652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.801652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:28.801652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:28.801652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.801652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:28.801652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:28.801652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.802283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:21:28.802283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:28.802283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:28.802283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:28.802283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:28.802283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:28.802283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:28.802283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:28.802283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:28.802283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:28.802283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:28.802283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:28.802283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:28.802283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:28.802283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:28.802283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:28.802283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:28.802283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:28.802283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:28.803023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:28.803023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.803023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:28.855781 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:28.856368 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:28.856524 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:28.856664 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:21:28.857393 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:28.857922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.858886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:28.858886 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:28.858886 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:28.858886 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:28.858886 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:28.858886 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:28.858886 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.858886 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:28.859974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.859974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:21:28.859974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:28.859974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:28.859974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:28.859974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:28.859974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:28.859974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:28.859974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:28.859974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:28.859974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:28.859974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:21:28.859974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:28.859974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:21:28.859974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:21:28.860280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:28.860280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:21:28.860280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:28.860280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:21:28.860280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:21:28.860280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:28.860280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:21:28.860280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.860280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:21:28.860882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.860882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:28.860882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:28.860882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:28.860882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.860882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:21:28.860882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:21:28.860882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:21:28.860882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:28.860882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:28.860882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:28.860882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:28.861151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:28.861151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:28.861151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:28.861151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:28.861151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:28.861151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:28.861151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:28.861151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:28.861151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:28.861151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:28.861151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:28.861151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:28.861151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:28.861151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:28.861413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:28.861413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:28.861413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.861413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:28.862564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.862564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:28.862564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:28.862564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:28.862564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:28.862564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:28.862564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.862564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:28.862564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:28.862564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.862564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:28.862564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:28.862564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.862564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:28.862564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:28.862564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.862910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:21:28.862910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:21:28.862910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:21:28.862910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:28.862910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:28.862910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:28.862910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:28.862910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:28.862910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:28.862910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:28.862910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:28.862910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:28.862910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:28.862910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:28.862910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:28.862910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:28.862910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:28.862910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:28.863260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:28.863260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:28.863260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:28.863260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:28.863260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:28.863260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.863260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:28.879661 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:28.880216 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:28.880282 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:28.880467 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:28.881385 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:28.882010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.884274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:28.884274 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:28.884274 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:28.884274 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:28.884274 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:28.884274 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:28.884274 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.884274 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:28.884274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.884274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:21:28.884274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:28.884274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:28.884274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:28.884274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:28.884274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:28.884274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:28.884274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:28.884274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:28.885593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:28.885593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:21:28.885593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:28.885593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:21:28.885593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:21:28.885593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:28.885593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:21:28.885593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:28.885593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:21:28.885593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:21:28.885593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:28.885593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:21:28.885593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.885593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:21:28.885593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.886021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:28.886021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:28.886021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:28.886021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.886021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:21:28.886021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:21:28.886021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:28.886021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:28.886021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:28.886021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:28.886021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:28.886021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:28.886021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:28.886386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:28.886386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:28.886386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:28.886386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:28.886386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:28.886386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:28.886386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:28.886386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:28.886386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:28.886386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:28.886386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:28.886386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:28.886386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:28.886854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.886854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:28.887455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.887455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:28.887455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:28.887455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:28.887455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:28.887455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:28.887455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.887455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:28.887455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:28.887455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.887455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:28.887455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:28.887455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.887455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:28.887455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:28.887455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.887455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:21:28.888528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:21:28.888528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:28.888528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:28.888528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:28.888528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:28.888528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:28.888528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:28.888528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:28.888528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:28.888528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:28.888528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:28.888528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:28.888528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:28.888528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:28.888528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:28.888528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:28.888528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:28.888528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:28.888528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:28.889148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:28.889148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:28.889148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.889148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:28.889148 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:28.889148 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:28.889148 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:28.889148 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:28.889744 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:28.890150 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:28.890501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.891198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:28.891198 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:28.891198 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:28.891198 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:28.891198 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:28.891198 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:28.891198 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.891198 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:28.891906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.891906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:28.891906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:28.891906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:28.891906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.891906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:21:28.891906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:28.891906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:28.891906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:28.891906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:28.891906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:28.891906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:28.892285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:28.892285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:28.892285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:28.892285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:28.892285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:28.892285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:28.892285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:28.892285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:28.892285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:28.892285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:28.892285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:28.892285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:28.892285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:28.892737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.892737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:28.892737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.892737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:28.892737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:28.892737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:28.892737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:28.892737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:28.892737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.892737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:28.892737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:28.892737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.892737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:28.892737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:28.892737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.893201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:28.893201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:28.893201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:28.893201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:21:28.893201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:28.893201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:28.893201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:28.893201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:28.893201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:28.893201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:28.893201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:28.893201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:28.893201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:28.893201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:28.893201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:28.893201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:28.893201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:28.893201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:28.893201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:28.893766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:28.893766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:28.893766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:28.893766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:28.893766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:28.893766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:33.562401 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:33.563285 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:33.563285 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:33.563720 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:33.604115 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:33.604519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.605382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:33.605382 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:33.605382 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:33.605382 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:33.605382 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:33.605382 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:33.605382 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.605382 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:33.606400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.606400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:33.606400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:33.606400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:33.606400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.606400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:21:33.606400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:33.606400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:33.606400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:33.606400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:33.606400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:33.606400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:33.606810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:33.606810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:33.606810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:33.606810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:33.606810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:33.606810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:33.606810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:33.606810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:33.606810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:33.606810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:33.606810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:33.606810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:33.606810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:33.607183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.607183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:33.607387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.607387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:33.607387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:33.607387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:33.607387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:33.607387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:33.607387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.607387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:33.607387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:33.607387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.607387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:33.607387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:33.607387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.607387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:33.607387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:33.607387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.607858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:21:33.607858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:33.607858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:33.607858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:33.607858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:33.607858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:33.607858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:33.607858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:33.607858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:33.607858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:33.607858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:33.607858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:33.607858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:33.607858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:33.607858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:33.607858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:33.607858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:33.607858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:33.607858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:33.608408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:33.608408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.608408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:33.798036 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:33.798425 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:33.798550 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:33.800998 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:33.802066 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:33.802372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.803188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:33.803188 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:33.803188 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:33.803188 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:33.803188 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:33.803188 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:33.803188 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.803188 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:33.804150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.804150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:33.804150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:33.804150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:33.804150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.804150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:21:33.804150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:33.804150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:33.804150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:33.804150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:33.804150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:33.804150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:33.804999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:33.804999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:33.804999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:33.804999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:33.804999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:33.804999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:33.804999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:33.804999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:33.804999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:33.804999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:33.804999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:33.804999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:33.804999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:33.805571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.805571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:33.805571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.805571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:33.805571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:33.805571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:33.805571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:33.805571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:33.805571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.805571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:33.805571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:33.805571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.805571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:33.805571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:33.805571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.806290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:33.806290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:33.806290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.806290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:21:33.806290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:33.806290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:33.806290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:33.806290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:33.806290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:33.806290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:33.806290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:33.806290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:33.806290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:33.806290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:33.806290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:33.806290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:33.806290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:33.806290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:33.806290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:33.807126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:33.807126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:33.807126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:33.807126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:33.807126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.807126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:33.856742 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:33.857356 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:33.857462 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:33.857572 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:21:33.858110 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:33.858288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.858816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:33.858816 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:33.858816 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:33.858816 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:33.858816 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:33.858816 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:33.858816 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.858816 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:33.859609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.859609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:21:33.859609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:33.859609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:33.859609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:33.859609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:33.859609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:33.859609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:33.859609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:33.859609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:33.859609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:33.859609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:21:33.859609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:33.859609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:21:33.859609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:21:33.860214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:33.860214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:21:33.860214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:33.860214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:21:33.860214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:21:33.860214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:33.860214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:21:33.860214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.860214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:21:33.860503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.860503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:33.860503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:33.860503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:33.860503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.860503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:21:33.860503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:21:33.860503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:21:33.860503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:33.860503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:33.860503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:33.860503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:33.860915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:33.860915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:33.860915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:33.860915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:33.860915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:33.860915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:33.860915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:33.860915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:33.860915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:33.860915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:33.860915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:33.860915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:33.860915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:33.860915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:33.861316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:33.861316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:33.861316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.861316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:33.861575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.861575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:33.861575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:33.861575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:33.861575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:33.861575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:33.861575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.861575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:33.861575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:33.861575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.861575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:33.861575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:33.861575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.861575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:33.861575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:33.861575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.862037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:21:33.862037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:21:33.862037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:21:33.862037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:33.862037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:33.862037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:33.862037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:33.862037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:33.862037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:33.862037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:33.862037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:33.862037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:33.862037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:33.862037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:33.862037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:33.862037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:33.862037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:33.862037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:33.862528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:33.862528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:33.862528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:33.862528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:33.862528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:33.862528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.862528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:33.880246 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:33.880722 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:33.880803 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:33.880924 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:33.881512 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:33.881869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.884396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:33.884396 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:33.884396 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:33.884396 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:33.884396 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:33.884396 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:33.884396 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.884396 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:33.884396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.884396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:21:33.884396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:33.884396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:33.884396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:33.884396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:33.884396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:33.884396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:33.884396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:33.884396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:33.884935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:33.884935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:21:33.884935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:33.884935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:21:33.884935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:21:33.884935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:33.884935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:21:33.884935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:33.884935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:21:33.884935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:21:33.884935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:33.884935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:21:33.884935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.884935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:21:33.884935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.885245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:33.885245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:33.885245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:33.885245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.885245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:21:33.885245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:21:33.885245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:33.885245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:33.885245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:33.885245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:33.885245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:33.885245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:33.885245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:33.885497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:33.885497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:33.885497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:33.885497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:33.885497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:33.885497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:33.885497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:33.885497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:33.885497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:33.885497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:33.885497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:33.885497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:33.885497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:33.885754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.885754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:33.886922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.886922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:33.886922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:33.886922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:33.886922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:33.886922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:33.886922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.886922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:33.886922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:33.886922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.886922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:33.886922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:33.886922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.886922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:33.886922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:33.886922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.886922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:21:33.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:21:33.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:33.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:33.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:33.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:33.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:33.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:33.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:33.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:33.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:33.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:33.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:33.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:33.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:33.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:33.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:33.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:33.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:33.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:33.887599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:33.887599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:33.887599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.887599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:33.888766 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:33.889571 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:33.889716 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:33.889868 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:33.890398 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:33.890781 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:33.891205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.891765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:33.891765 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:33.891765 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:33.891765 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:33.891765 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:33.891765 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:33.891765 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.891765 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:33.892661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.892661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:33.892661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:33.892661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:33.892661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.892661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:21:33.892661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:33.892661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:33.892661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:33.892661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:33.892661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:33.892661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:33.893209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:33.893209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:33.893209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:33.893209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:33.893209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:33.893209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:33.893209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:33.893209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:33.893209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:33.893209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:33.893209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:33.893209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:33.893209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:33.893774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.893774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:33.893774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.893774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:33.893774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:33.893774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:33.893774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:33.893774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:33.893774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.893774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:33.893774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:33.893774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.893774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:33.893774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:33.893774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.894406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:33.894406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:33.894406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:33.894406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:21:33.894406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:33.894406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:33.894406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:33.894406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:33.894406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:33.894406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:33.894406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:33.894406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:33.894406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:33.894406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:33.894406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:33.894406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:33.894406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:33.894406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:33.894406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:33.895024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:33.895024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:33.895024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:33.895024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:33.895024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:33.895024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:38.563309 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:38.563920 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:38.563920 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:38.565380 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:38.602185 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:38.602516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.603625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:38.603625 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:38.603625 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:38.603625 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:38.603625 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:38.603625 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:38.603625 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.603625 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:38.604749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.604749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:38.604749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:38.604749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:38.604749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.604749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:21:38.604749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:38.604749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:38.604749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:38.604749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:38.604749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:38.604749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:38.605121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:38.605121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:38.605121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:38.605121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:38.605121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:38.605121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:38.605121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:38.605121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:38.605121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:38.605121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:38.605121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:38.605121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:38.605121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:38.605478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.605478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:38.605969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.605969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:38.605969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:38.605969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:38.605969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:38.605969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:38.605969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.605969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:38.605969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:38.605969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.605969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:38.605969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:38.605969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.605969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:38.605969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:38.605969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.606426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:21:38.606426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:38.606426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:38.606426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:38.606426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:38.606426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:38.606426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:38.606426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:38.606426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:38.606426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:38.606426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:38.606426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:38.606426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:38.606426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:38.606426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:38.606426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:38.606426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:38.606426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:38.606426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:38.607154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:38.607154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.607154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:38.798565 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:38.799270 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:38.799493 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:38.800577 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:38.800943 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:38.802495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.803486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:38.803486 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:38.803486 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:38.803486 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:38.803486 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:38.803486 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:38.803486 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.803486 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:38.805233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.805233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:38.805233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:38.805233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:38.805233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.805233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:21:38.805233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:38.805233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:38.805233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:38.805233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:38.805233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:38.805233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:38.806115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:38.806115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:38.806115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:38.806115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:38.806115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:38.806115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:38.806115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:38.806115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:38.806115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:38.806115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:38.806115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:38.806115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:38.806115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:38.806649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.806649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:38.808234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.808234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:38.808234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:38.808234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:38.808234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:38.808234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:38.808234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.808234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:38.808234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:38.808234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.808234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:38.808234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:38.808234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.808234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:38.808234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:38.808234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:21:38.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:38.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:38.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:38.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:38.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:38.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:38.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:38.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:38.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:38.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:38.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:38.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:38.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:38.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:38.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:38.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:38.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:38.809944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:38.810513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:38.810513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.810513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:38.857450 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:38.858182 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:38.858386 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:38.858607 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:21:38.859289 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:38.859612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.860275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:38.860275 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:38.860275 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:38.860275 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:38.860275 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:38.860275 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:38.860275 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.860275 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:38.861141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.861141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:21:38.861141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:38.861141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:38.861141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:38.861141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:38.861141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:38.861141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:38.861141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:38.861141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:38.861141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:38.861141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:21:38.861141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:38.861141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:21:38.861141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:21:38.864168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:38.864168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:21:38.864168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:38.864168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:21:38.864168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:21:38.864168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:38.864168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:21:38.864168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.864168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:21:38.864168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.864168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:38.864168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:38.864168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:38.864168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.864818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:21:38.864818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:21:38.864818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:21:38.864818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:38.864818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:38.864818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:38.864818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:38.864818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:38.864818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:38.864818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:38.864818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:38.864818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:38.864818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:38.865402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:38.865402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:38.865402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:38.865402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:38.865402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:38.865402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:38.865402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:38.865402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:38.865402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:38.865402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:38.865402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.865402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:38.866017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.866017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:38.866017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:38.866017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:38.866017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:38.866017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:38.866017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.866017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:38.866017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:38.866017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.866017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:38.866017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:38.866017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.866017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:38.866017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:38.866017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.866694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:21:38.866694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:21:38.866694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:21:38.866694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:38.866694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:38.866694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:38.866694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:38.866694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:38.866694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:38.866694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:38.866694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:38.866694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:38.866694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:38.866694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:38.866694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:38.866694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:38.866694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:38.866694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:38.867210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:38.867210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:38.867210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:38.867210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:38.867210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:38.867210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.867210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:38.880812 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:38.881742 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:38.881742 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:38.881742 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:38.882584 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:38.883544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.885295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:38.885295 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:38.885295 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:38.885295 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:38.885295 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:38.885295 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:38.885295 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.885295 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:38.887337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.887337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:21:38.887337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:38.887337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:38.887337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:38.887337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:38.887337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:38.887337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:38.887337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:38.887337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:38.887337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:38.887337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:21:38.887337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:38.887337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:21:38.887337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:21:38.890251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:38.890251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:21:38.890251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:38.890251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:21:38.890251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:21:38.890251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:38.890251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:21:38.890251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.890251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:21:38.890251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.890251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:38.890251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:38.890251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:38.890251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.890979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:21:38.890979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:21:38.890979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:38.890979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:38.890979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:38.890979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:38.890979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:38.890979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:38.890979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:38.890979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:38.890979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:38.890979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:38.890979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:38.891345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:38.891345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:38.891345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:38.891345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:38.891345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:38.891345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:38.891345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:38.891345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:38.891345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:38.891345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.891345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:38.891345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.891936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:38.891936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:38.891936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:38.891936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:38.891936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:38.891936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.891936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:38.891936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:38.891936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.891936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:38.891936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:38.891936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.891936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:38.891936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:38.891936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.891936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:21:38.891936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:21:38.891936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:38.891936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:38.892556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:38.892556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:38.892556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:38.892556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:38.892556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:38.892556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:38.892556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:38.892556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:38.892556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:38.892556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:38.892556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:38.892556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:38.892556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:38.892556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:38.892556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:38.892556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:38.892556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:38.892556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:38.892556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.892556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:38.893207 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:38.893207 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:38.893207 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:38.893207 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:38.893207 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:38.893207 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:38.893594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.895261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:38.895261 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:38.895261 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:38.895261 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:38.895261 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:38.895261 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:38.895261 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.895261 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:38.895913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.895913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:38.895913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:38.895913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:38.895913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.895913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:21:38.895913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:38.895913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:38.895913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:38.895913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:38.895913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:38.895913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:38.896311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:38.896311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:38.896311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:38.896311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:38.896311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:38.896311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:38.896311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:38.896311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:38.896311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:38.896311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:38.896311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:38.896311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:38.896311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:38.896728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.896728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:38.896728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.896728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:38.896728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:38.896728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:38.896728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:38.896728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:38.896728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.896728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:38.896728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:38.896728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.896728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:38.896728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:38.896728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.897265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:38.897265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:38.897265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:38.897265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:21:38.897265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:38.897265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:38.897265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:38.897265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:38.897265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:38.897265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:38.897265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:38.897265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:38.897265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:38.897265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:38.897265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:38.897265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:38.897265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:38.897265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:38.897265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:38.897926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:38.897926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:38.897926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:38.897926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:38.897926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:38.897926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:43.563789 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:43.565243 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:43.565243 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:43.565985 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:43.601444 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:43.601718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.602459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:43.602459 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:43.602459 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:43.602459 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:43.602459 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:43.602459 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:43.602459 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.602459 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:43.603508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.603508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:43.603508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:43.603508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:43.603508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.603508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:21:43.603508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:43.603508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:43.603508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:43.603508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:43.603508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:43.603508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:43.603898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:43.603898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:43.603898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:43.603898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:43.603898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:43.603898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:43.603898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:43.603898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:43.603898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:43.603898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:43.603898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:43.603898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:43.603898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:43.604259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.604259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:43.604593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.604593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:43.604593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:43.604593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:43.604593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:43.604593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:43.604593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.604593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:43.604593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:43.604593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.604593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:43.604593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:43.604593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.604593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:43.604593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:43.604593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.605751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:21:43.605751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:43.605751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:43.605751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:43.605751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:43.605751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:43.605751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:43.605751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:43.605751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:43.605751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:43.605751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:43.605751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:43.605751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:43.605751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:43.605751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:43.605751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:43.605751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:43.605751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:43.605751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:43.606869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:43.606869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.606869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:43.799260 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:43.799767 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:43.799767 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:43.800505 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:43.800731 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:43.800943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.801300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:43.801300 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:43.801300 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:43.801300 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:43.801300 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:43.801300 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:43.801300 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.801300 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:43.802845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.802845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:43.802845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:43.802845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:43.802845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.802845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:21:43.802845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:43.802845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:43.802845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:43.802845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:43.802845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:43.802845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:43.804082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:43.804082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:43.804082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:43.804082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:43.804082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:43.804082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:43.804082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:43.804082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:43.804082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:43.804082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:43.804082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:43.804082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:43.804082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:43.804597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.804597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:43.804597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.804597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:43.804597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:43.804597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:43.804597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:43.804597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:43.804597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.804597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:43.804597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:43.804597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.804597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:43.804597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:43.804597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.806137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:43.806137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:43.806137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.806137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:21:43.806137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:43.806137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:43.806137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:43.806137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:43.806137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:43.806137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:43.806137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:43.806137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:43.806137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:43.806137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:43.806137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:43.806137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:43.806137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:43.806137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:43.806137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:43.806830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:43.806830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:43.806830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:43.806830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:43.806830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.806830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:43.858548 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:43.859022 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:43.859022 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:43.859347 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:21:43.859616 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:43.859872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.860283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:43.860283 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:43.860283 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:43.860283 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:43.860283 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:43.860283 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:43.860283 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.860283 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:43.860931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.860931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:21:43.860931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:43.860931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:43.860931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:43.860931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:43.860931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:43.860931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:43.860931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:43.860931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:43.860931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:43.860931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:21:43.860931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:43.860931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:21:43.860931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:21:43.861312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:43.861312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:21:43.861312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:43.861312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:21:43.861312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:21:43.861312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:43.861312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:21:43.861312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.861312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:21:43.861532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.861532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:43.861532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:43.861532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:43.861532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.861532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:21:43.861532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:21:43.861532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:21:43.861532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:43.861532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:43.861532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:43.861532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:43.861850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:43.861850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:43.861850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:43.861850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:43.861850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:43.861850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:43.861850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:43.861850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:43.861850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:43.861850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:43.861850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:43.861850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:43.861850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:43.861850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:43.862160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:43.862160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:43.862160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.862160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:43.862264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.862264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:43.862264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:43.862264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:43.862264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:43.862264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:43.862264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.862264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:43.862264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:43.862264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.862264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:43.862264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:43.862264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.862264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:43.862264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:43.862264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.862665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:21:43.862665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:21:43.862665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:21:43.862665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:43.862665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:43.862665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:43.862665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:43.862665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:43.862665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:43.862665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:43.862665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:43.862665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:43.862665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:43.862665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:43.862665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:43.862665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:43.862665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:43.862665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:43.863083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:43.863083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:43.863083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:43.863083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:43.863083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:43.863083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.863083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:43.882071 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:43.882522 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:43.882755 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:43.882954 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:43.883459 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:43.883668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.884247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:43.884247 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:43.884247 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:43.884247 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:43.884247 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:43.884247 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:43.884247 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.884247 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:43.886817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.886817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:21:43.886817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:43.886817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:43.886817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:43.886817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:43.886817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:43.886817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:43.886817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:43.886817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:43.886817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:43.886817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:21:43.886817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:43.886817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:21:43.886817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:21:43.887192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:43.887192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:21:43.887192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:43.887192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:21:43.887192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:21:43.887192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:43.887192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:21:43.887192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.887192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:21:43.889544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.889544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:43.889544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:43.889544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:43.889544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.889544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:21:43.889544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:21:43.889544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:43.889544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:43.889544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:43.889544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:43.889887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:43.889887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:43.889887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:43.889887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:43.889887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:43.889887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:43.889887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:43.889887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:43.889887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:43.889887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:43.889887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:43.889887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:43.889887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:43.889887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:43.890291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:43.890291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:43.890291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.890291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:43.891228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.891228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:43.891228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:43.891228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:43.891228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:43.891228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:43.891228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.891228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:43.891228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:43.891228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.891228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:43.891228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:43.891228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.891228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:43.891228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:43.891228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.891228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:21:43.892129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:21:43.892129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:43.892129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:43.892129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:43.892129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:43.892129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:43.892129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:43.892129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:43.892129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:43.892129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:43.892129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:43.892129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:43.892129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:43.892129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:43.892129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:43.892129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:43.892129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:43.892129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:43.892129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:43.892533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:43.892533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:43.892533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.892533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:43.892533 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:43.892533 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:43.892533 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:43.892533 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:43.892762 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:43.893014 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:43.893014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.893342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:43.893342 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:43.893342 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:43.893342 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:43.893342 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:43.893342 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:43.893342 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.893342 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:43.893946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.893946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:43.893946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:43.893946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:43.893946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.893946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:21:43.893946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:43.893946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:43.893946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:43.893946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:43.893946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:43.893946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:43.894632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:43.894632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:43.894632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:43.894632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:43.894632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:43.894632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:43.894632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:43.894632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:43.894632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:43.894632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:43.894632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:43.894632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:43.894632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:43.894929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.894929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:43.894929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.894929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:43.894929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:43.894929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:43.894929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:43.894929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:43.894929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.894929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:43.894929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:43.894929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.894929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:43.894929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:43.894929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.895265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:43.895265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:43.895265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:43.895265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:21:43.895265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:43.895265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:43.895265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:43.895265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:43.895265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:43.895265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:43.895265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:43.895265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:43.895265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:43.895265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:43.895265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:43.895265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:43.895265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:43.895265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:43.895265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:43.895668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:43.895668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:43.895668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:43.895668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:43.895668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:43.895668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:48.565707 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:48.565707 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:48.565707 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:48.566393 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:48.600709 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:48.600709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.601008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:48.601008 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:48.601008 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:48.601008 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:48.601008 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:48.601008 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:48.601008 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.601008 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:48.602064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.602064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:48.602064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:48.602064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:48.602064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.602064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:21:48.602064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:48.602064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:48.602064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:48.602064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:48.602064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:48.602064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:48.603577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:48.603577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:48.603577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:48.603577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:48.603577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:48.603577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:48.603577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:48.603577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:48.603577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:48.603577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:48.603577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:48.603577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:48.603577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:48.604994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.604994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:48.604994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.604994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:48.604994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:48.604994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:48.604994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:48.604994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:48.604994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.604994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:48.604994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:48.604994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.604994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:48.604994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:48.604994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.607191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:48.607191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:48.607191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.607191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:21:48.607191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:48.607191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:48.607191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:48.607191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:48.607191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:48.607191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:48.607191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:48.607191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:48.607191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:48.607191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:48.607191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:48.607191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:48.607191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:48.607191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:48.607191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:48.607593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:48.607593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:48.607593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:48.607593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:48.607593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.607593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:48.799902 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:48.800338 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:48.800463 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:48.801312 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:48.801485 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:48.801600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.802155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:48.802155 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:48.802155 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:48.802155 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:48.802155 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:48.802155 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:48.802155 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.802155 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:48.803220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.803220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:48.803220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:48.803220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:48.803220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.803220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:21:48.803220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:48.803220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:48.803220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:48.803220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:48.803220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:48.803220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:48.803936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:48.803936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:48.803936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:48.803936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:48.803936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:48.803936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:48.803936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:48.803936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:48.803936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:48.803936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:48.803936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:48.803936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:48.803936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:48.804175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.804175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:48.804462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.804462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:48.804462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:48.804462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:48.804462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:48.804462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:48.804462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.804462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:48.804462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:48.804462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.804462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:48.804462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:48.804462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.804462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:48.804462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:48.804462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.804797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:21:48.804797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:48.804797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:48.804797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:48.804797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:48.804797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:48.804797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:48.804797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:48.804797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:48.804797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:48.804797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:48.804797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:48.804797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:48.804797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:48.804797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:48.804797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:48.804797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:48.804797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:48.804797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:48.805146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:48.805146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.805146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:48.859019 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:48.860706 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:48.860706 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:48.860706 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:21:48.860706 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:48.860706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.861450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:48.861450 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:48.861450 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:48.861450 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:48.861450 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:48.861450 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:48.861450 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.861450 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:48.862352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.862352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:21:48.862352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:48.862352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:48.862352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:48.862352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:48.862352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:48.862352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:48.862352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:48.862352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:48.862352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:48.862352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:21:48.862352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:48.862352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:21:48.862352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:21:48.862719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:48.862719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:21:48.862719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:48.862719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:21:48.862719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:21:48.862719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:48.862719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:21:48.862719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.862719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:21:48.863182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.863182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:48.863182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:48.863182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:48.863182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.863182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:21:48.863182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:21:48.863182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:21:48.863182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:48.863182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:48.863182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:48.863182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:48.863508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:48.863508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:48.863508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:48.863508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:48.863508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:48.863508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:48.863508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:48.863508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:48.863508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:48.863508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:48.863508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:48.863508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:48.863508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:48.863508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:48.863780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:48.863780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:48.863780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.863780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:48.864297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.864297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:48.864297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:48.864297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:48.864297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:48.864297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:48.864297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.864297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:48.864297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:48.864297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.864297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:48.864297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:48.864297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.864297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:48.864297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:48.864297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.864637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:21:48.864637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:21:48.864637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:21:48.864637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:48.864637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:48.864637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:48.864637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:48.864637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:48.864637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:48.864637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:48.864637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:48.864637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:48.864637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:48.864637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:48.864637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:48.864637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:48.864637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:48.864637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:48.865025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:48.865025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:48.865025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:48.865025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:48.865025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:48.865025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.865025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:48.883713 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:48.883883 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:48.884004 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:48.884157 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:48.884703 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:48.885032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.885779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:48.885779 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:48.885779 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:48.885779 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:48.885779 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:48.885779 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:48.885779 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.885779 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:48.887016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.887016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:21:48.887016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:48.887016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:48.887016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:48.887016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:48.887016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:48.887016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:48.887016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:48.887016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:48.887016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:48.887016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:21:48.887016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:48.887016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:21:48.887016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:21:48.887313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:48.887313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:21:48.887313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:48.887313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:21:48.887313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:21:48.887313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:48.887313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:21:48.887313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.887313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:21:48.890438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.890438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:48.890438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:48.890438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:48.890438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.890438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:21:48.890438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:21:48.890438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:48.890438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:48.890438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:48.890438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:48.890733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:48.890733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:48.890733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:48.890733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:48.890733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:48.890733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:48.890733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:48.890733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:48.890733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:48.890733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:48.890733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:48.890733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:48.890733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:48.890733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:48.891013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:48.891013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:48.891013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.891013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:48.891395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.891395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:48.891395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:48.891395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:48.891395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:48.891395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:48.891395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.891395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:48.891395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:48.891395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.891395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:48.891395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:48.891395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.891395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:48.891395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:48.891395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.891395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:21:48.891792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:21:48.891792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:48.891792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:48.891792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:48.891792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:48.891792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:48.891792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:48.891792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:48.891792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:48.891792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:48.891792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:48.891792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:48.891792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:48.891792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:48.891792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:48.891792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:48.891792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:48.891792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:48.891792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:48.892307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:48.892307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:48.892307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.892307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:48.892794 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:48.892794 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:48.893133 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:48.893133 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:48.893483 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:48.893832 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:48.894199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.894653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:48.894653 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:48.894653 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:48.894653 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:48.894653 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:48.894653 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:48.894653 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.894653 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:48.895416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.895416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:48.895416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:48.895416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:48.895416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.895416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:21:48.895416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:48.895416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:48.895416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:48.895416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:48.895416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:48.895416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:48.896030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:48.896030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:48.896030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:48.896030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:48.896030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:48.896030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:48.896030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:48.896030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:48.896030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:48.896030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:48.896030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:48.896030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:48.896030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:48.896528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.896528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:48.896528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.896528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:48.896528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:48.896528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:48.896528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:48.896528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:48.896528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.896528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:48.896528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:48.896528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.896528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:48.896528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:48.896528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.896954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:48.896954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:48.896954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:48.896954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:21:48.896954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:48.896954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:48.896954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:48.896954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:48.896954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:48.896954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:48.896954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:48.896954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:48.896954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:48.896954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:48.896954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:48.896954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:48.896954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:48.896954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:48.896954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:48.897441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:48.897441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:48.897441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:48.897441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:48.897441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:48.897441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:53.564938 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:53.565604 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:53.565754 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:53.566988 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:53.603161 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:53.603363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.603936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:53.603936 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:53.603936 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:53.603936 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:53.603936 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:53.603936 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:53.603936 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.603936 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:53.604742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.604742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:53.604742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:53.604742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:53.604742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.604742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:21:53.604742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:53.604742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:53.604742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:53.604742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:53.604742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:53.604742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:53.605110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:53.605110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:53.605110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:53.605110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:53.605110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:53.605110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:53.605110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:53.605110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:53.605110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:53.605110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:53.605110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:53.605110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:53.605110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:53.605388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.605388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:53.605767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.605767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:53.605767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:53.605767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:53.605767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:53.605767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:53.605767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.605767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:53.605767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:53.605767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.605767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:53.605767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:53.605767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.605767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:53.605767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:53.605767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.606130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:21:53.606130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:53.606130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:53.606130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:53.606130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:53.606130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:53.606130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:53.606130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:53.606130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:53.606130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:53.606130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:53.606130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:53.606130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:53.606130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:53.606130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:53.606130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:53.606130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:53.606130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:53.606130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:53.606536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:53.606536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.606536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:53.800388 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:53.800960 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:53.801092 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:53.802273 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:53.802273 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:53.802421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.804552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:53.804552 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:53.804552 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:53.804552 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:53.804552 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:53.804552 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:53.804552 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.804552 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:53.806063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.806063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:53.806063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:53.806063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:53.806063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.806063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:21:53.806063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:53.806063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:53.806063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:53.806063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:53.806063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:53.806063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:53.806740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:53.806740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:53.806740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:53.806740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:53.806740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:53.806740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:53.806740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:53.806740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:53.806740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:53.806740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:53.806740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:53.806740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:53.806740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:53.807396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.807396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:53.807396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.807396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:53.807396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:53.807396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:53.807396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:53.807396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:53.807396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.807396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:53.807396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:53.807396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.807396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:53.807396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:53.807396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.808128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:53.808128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:53.808128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.808128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:21:53.808128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:53.808128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:53.808128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:53.808128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:53.808128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:53.808128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:53.808128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:53.808128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:53.808128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:53.808128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:53.808128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:53.808128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:53.808128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:53.808128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:53.808128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:53.809050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:53.809050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:53.809050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:53.809050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:53.809050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.809050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:53.859587 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:53.860720 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:53.860720 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:53.860720 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:21:53.861068 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:53.861247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.861789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:53.861789 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:53.861789 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:53.861789 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:53.861789 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:53.861789 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:53.861789 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.861789 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:53.864953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.864953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:21:53.864953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:53.864953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:53.864953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:53.864953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:53.864953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:53.864953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:53.864953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:53.864953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:53.864953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:53.864953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:21:53.864953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:53.864953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:21:53.864953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:21:53.865441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:53.865441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:21:53.865441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:53.865441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:21:53.865441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:21:53.865441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:53.865441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:21:53.865441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.865441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:21:53.865441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.865441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:53.865441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:53.865441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:53.865441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:21:53.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:21:53.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:21:53.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:53.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:53.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:53.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:53.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:53.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:53.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:53.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:53.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:53.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:53.866065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:53.866065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:53.866065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:53.866065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:53.866065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:53.866065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:53.866065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:53.866065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:53.866065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:53.866065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:53.866065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.866065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:53.866308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.866308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:53.866308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:53.866308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:53.866308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:53.866308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:53.866308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.866308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:53.866308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:53.866308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.866308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:53.866308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:53.866308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.866308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:53.866308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:53.866308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.866594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:21:53.866594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:21:53.866594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:21:53.866594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:53.866594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:53.866594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:53.866594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:53.866594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:53.866594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:53.866594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:53.866594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:53.866594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:53.866594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:53.866594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:53.866594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:53.866594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:53.866594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:53.866594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:53.866950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:53.866950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:53.866950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:53.866950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:53.866950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:53.866950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.866950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:53.883850 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:53.884263 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:53.884419 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:53.884579 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:53.885765 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:53.885765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.886102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:53.886102 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:53.886102 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:53.886102 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:53.886102 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:53.886102 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:53.886102 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.886102 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:53.886807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.886807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:21:53.886807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:53.886807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:53.886807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:53.886807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:53.886807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:53.886807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:53.886807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:53.886807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:53.886807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:53.886807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:21:53.886807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:53.886807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:21:53.886807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:21:53.887567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:53.887567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:21:53.887567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:53.887567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:21:53.887567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:21:53.887567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:53.887567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:21:53.887567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.887567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:21:53.888043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.888043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:53.888043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:53.888043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:53.888043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.888043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:21:53.888043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:21:53.888043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:53.888043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:53.888043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:53.888043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:53.888631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:53.888631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:53.888631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:53.888631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:53.888631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:53.888631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:53.888631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:53.888631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:53.888631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:53.888631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:53.888631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:53.888631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:53.888631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:53.888631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:53.889286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:53.889286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:53.889286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.889286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:53.889286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.889286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:53.889286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:53.889286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:53.889286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:53.889286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:53.889286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.889286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:53.889286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:53.889286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.892113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:53.892113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:53.892113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.892113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:53.892113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:53.892113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.892113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:21:53.892113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:21:53.892113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:53.892113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:53.892113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:53.892113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:53.892113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:53.892113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:53.892113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:53.892113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:53.892113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:53.892113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:53.892113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:53.893427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:53.893427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:53.893427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:53.893427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:53.893427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:53.893427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:53.893427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:53.893427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:53.893427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:53.893427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.893427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:53.893427 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:53.896182 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:53.896281 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:53.896579 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:53.897412 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:53.897692 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:53.897913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.898553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:53.898553 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:53.898553 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:53.898553 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:53.898553 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:53.898553 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:53.898553 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.898553 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:53.899666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.899666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:53.899666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:53.899666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:53.899666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.899666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:21:53.899666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:53.899666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:53.899666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:53.899666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:53.899666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:53.899666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:53.899959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:53.899959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:53.899959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:53.899959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:53.899959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:53.899959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:53.899959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:53.899959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:53.899959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:53.899959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:53.899959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:53.899959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:53.899959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:53.900252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.900252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:53.900899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.900899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:53.900899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:53.900899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:53.900899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:53.900899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:53.900899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.900899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:53.900899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:53.900899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.900899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:53.900899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:53.900899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.900899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:53.900899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:53.900899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:53.901299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:21:53.901299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:53.901299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:53.901299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:53.901299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:53.901299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:53.901299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:53.901299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:53.901299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:53.901299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:53.901299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:53.901299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:53.901299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:53.901299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:53.901299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:53.901299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:53.901299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:53.901299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:53.901299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:53.901722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:53.901722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:53.901722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:58.566080 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:58.567418 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:58.567515 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:58.568188 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:58.617046 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:58.617354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.619688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:58.619688 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:58.619688 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:58.619688 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:58.619688 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:58.619688 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:58.619688 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.619688 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:58.620464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.051 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.620464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:58.620464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:58.620464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:58.620464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.620464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:21:58.620464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:58.620464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:58.620464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:58.620464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:58.620464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:58.620464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:58.620864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:58.620864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:58.620864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:58.620864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:58.620864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:58.620864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:58.620864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:58.620864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:58.620864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:58.620864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:58.620864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:58.620864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:58.620864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:58.621242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.621242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:58.622576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.622576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:58.622576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:58.622576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:58.622576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:58.622576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:58.622576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.622576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:58.622576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:58.622576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.622576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:58.622576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:58.622576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.622576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:58.622576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:58.622576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.623099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:21:58.623099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:58.623099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:58.623099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:58.623099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:58.623099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:58.623099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:58.623099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:58.623099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:58.623099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:58.623099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:58.623099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:58.623099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:58.623099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:58.623099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:58.623099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:58.623099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:58.623099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:58.623099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:58.623633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:58.623633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.623633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:58.801118 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:58.801790 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:58.802109 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:58.802833 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:58.802989 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:58.803116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.803652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:58.803652 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:58.803652 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:58.803652 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:58.803652 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:58.803652 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:58.803652 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.803652 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:58.806169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.806169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:58.806169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:58.806169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:58.806169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.806169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:21:58.806169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:58.806169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:58.806169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:58.806169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:58.806169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:58.806169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:58.806547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:58.806547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:58.806547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:58.806547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:58.806547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:58.806547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:58.806547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:58.806547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:58.806547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:58.806547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:58.806547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:58.806547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:58.806547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:58.806935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.806935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:58.806935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.806935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:58.806935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:58.806935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:58.806935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:58.806935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:58.806935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.806935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:58.806935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:58.806935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.806935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:58.806935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:58.806935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.807368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:58.807368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:58.807368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.807368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:21:58.807368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:58.807368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:58.807368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:58.807368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:58.807368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:58.807368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:58.807368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:58.807368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:58.807368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:58.807368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:58.807368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:58.807368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:58.807368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:58.807368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:58.807368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:58.807882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:58.807882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:58.807882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:58.807882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:58.807882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.807882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:58.860553 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:58.861180 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:58.861180 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:58.861180 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:21:58.861622 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:58.861838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.865183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:58.865183 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:58.865183 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:58.865183 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:58.865183 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:58.865183 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:58.865183 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.865183 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:58.865183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.865183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:21:58.865183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:58.865183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:58.865183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:58.865183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:58.865183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:58.865183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:58.865183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:58.865183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:58.866123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:58.866123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:21:58.866123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:58.866123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:21:58.866123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:21:58.866123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:58.866123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:21:58.866123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:58.866123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:21:58.866123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:21:58.866123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:58.866123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:21:58.866123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.866123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:21:58.866123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.866925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:58.866925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:58.866925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:58.866925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.866925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:21:58.866925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:21:58.866925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:21:58.866925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:58.866925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:58.866925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:58.866925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:58.866925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:58.866925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:58.867433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:58.867433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:58.867433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:58.867433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:58.867433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:58.867433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:58.867433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:58.867433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:58.867433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:58.867433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:58.867433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:58.867433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:58.867433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:58.867433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:58.868026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.868026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:58.869585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.869585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:58.869585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:58.869585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:58.869585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:58.869585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:58.869585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.869585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:58.869585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:58.869585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.869585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:58.869585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:58.869585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.869585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:58.869585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:58.869585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.870225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:21:58.870225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:21:58.870225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:21:58.870225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:58.870225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:58.870225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:58.870225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:58.870225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:58.870225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:58.870225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:58.870225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:58.870225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:58.870225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:58.870225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:58.870225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:58.870225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:58.870225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:58.870225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:58.870744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:58.870744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:58.870744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:58.870744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:58.870744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:58.870744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.870744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:58.884379 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:58.884922 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:58.885204 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:58.885492 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:58.886032 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:58.886278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.889461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:58.889461 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:58.889461 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:58.889461 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:58.889461 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:58.889461 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:58.889461 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.889461 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:58.891063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.891063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:21:58.891063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:58.891063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:58.891063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:58.891063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:58.891063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:58.891063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:58.891063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:58.891063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:21:58.891063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:58.891063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:21:58.891063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:58.891063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:21:58.891063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:21:58.891540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:58.891540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:21:58.891540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:58.891540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:21:58.891540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:21:58.891540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:58.891540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:21:58.891540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.891540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:21:58.892949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.892949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:58.892949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:58.892949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:58.892949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.892949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:21:58.892949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:21:58.892949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:58.892949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:58.892949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:58.892949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:58.893406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:58.893406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:58.893406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:58.893406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:58.893406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:58.893406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:58.893406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:58.893406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:58.893406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:58.893406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:58.893406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:58.893406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:58.893406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:58.893406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:58.893842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:58.893842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:58.893842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.893842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:58.894801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.894801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:58.894801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:58.894801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:58.894801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:58.894801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:58.894801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.894801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:58.894801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:58.894801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.894801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:58.894801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:58.894801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.894801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:58.894801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:58.894801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.894801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:21:58.895604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:21:58.895604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:21:58.895604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:21:58.895604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:58.895604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:58.895604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:58.895604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:58.895604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:58.895604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:58.895604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:58.895604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:58.895604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:58.895604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:58.895604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:58.895604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:58.895604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:58.895604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:58.895604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:58.895604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:58.897327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:58.897327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:58.897327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.897327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:21:58.897327 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:21:58.897327 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:21:58.897327 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:21:58.897327 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:21:58.897854 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:58.898155 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:21:58.898300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.898807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:21:58.898807 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:58.898807 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:21:58.898807 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:21:58.898807 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:58.898807 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:21:58.898807 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.898807 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:21:58.899420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.899420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:21:58.899420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:58.899420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:21:58.899420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.899420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:21:58.899420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:58.899420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:58.899420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:58.899420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:58.899420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:58.899420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:58.900153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:21:58.900153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:58.900153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:21:58.900153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:58.900153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:21:58.900153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:21:58.900153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:58.900153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:21:58.900153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:58.900153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:21:58.900153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:21:58.900153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:58.900153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:21:58.901121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.901121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:21:58.901121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.901121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:21:58.901121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:21:58.901121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:21:58.901121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:21:58.901121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:21:58.901121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.901121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:21:58.901121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:21:58.901121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.901121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:21:58.901121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:21:58.901121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.901827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:21:58.901827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:21:58.901827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:21:58.901827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:21:58.901827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:21:58.901827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:58.901827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:58.901827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:21:58.901827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:21:58.901827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:21:58.901827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:21:58.901827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:21:58.901827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:21:58.901827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:21:58.901827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:21:58.901827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:21:58.901827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:21:58.901827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:21:58.901827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:21:58.902967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:21:58.902967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:21:58.902967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:21:58.902967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:21:58.902967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:21:58.902967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:03.567671 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:03.568345 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:03.568345 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:03.569080 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:03.570542 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=471,neutron:UPDATE=2 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:22:03.603273 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:03.603273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.603927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:03.603927 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:03.603927 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:03.603927 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:03.603927 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:03.603927 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:03.603927 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.603927 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:03.604921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.604921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:03.604921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:03.604921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:03.604921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.604921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:22:03.604921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:03.604921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:03.604921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:03.604921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:03.604921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:03.604921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:03.605621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:03.605621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:03.605621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:03.605621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:03.605621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:03.605621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:03.605621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:03.605621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:03.605621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:03.605621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:03.605621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:03.605621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:03.605621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:03.606670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.606670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:03.606670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.606670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:03.606670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:03.606670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:03.606670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:03.606670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:03.606670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.606670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:03.606670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:03.606670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.606670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:03.606670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:03.606670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.607401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:03.607401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:03.607401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.607401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:22:03.607401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:03.607401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:03.607401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:03.607401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:03.607401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:03.607401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:03.607401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:03.607401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:03.607401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:03.607401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:03.607401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:03.607401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:03.607401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:03.607401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:03.607401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:03.608332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:03.608332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:03.608332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:03.608332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:03.608332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.608332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:03.802244 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:03.802708 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:03.802891 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:03.803769 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:03.803965 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:03.804086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.804653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:03.804653 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:03.804653 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:03.804653 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:03.804653 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:03.804653 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:03.804653 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.804653 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:03.807111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.807111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:03.807111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:03.807111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:03.807111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.807111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:22:03.807111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:03.807111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:03.807111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:03.807111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:03.807111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:03.807111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:03.807519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:03.807519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:03.807519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:03.807519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:03.807519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:03.807519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:03.807519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:03.807519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:03.807519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:03.807519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:03.807519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:03.807519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:03.807519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:03.807947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.807947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:03.809188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.809188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:03.809188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:03.809188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:03.809188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:03.809188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:03.809188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.809188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:03.809188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:03.809188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.809188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:03.809188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:03.809188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.809188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:03.809188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:03.809188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.809885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:22:03.809885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:03.809885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:03.809885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:03.809885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:03.809885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:03.809885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:03.809885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:03.809885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:03.809885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:03.809885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:03.809885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:03.809885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:03.809885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:03.809885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:03.809885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:03.809885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:03.809885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:03.809885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:03.810503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:03.810503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.810503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:03.860939 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:03.861365 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:03.861454 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:03.861630 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:22:03.862253 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:03.862594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.864298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:03.864298 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:03.864298 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:03.864298 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:03.864298 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:03.864298 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:03.864298 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.864298 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:03.865645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.865645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:22:03.865645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:03.865645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:03.865645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:03.865645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:03.865645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:03.865645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:03.865645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:03.865645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:03.865645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:03.865645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:22:03.865645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:03.865645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:22:03.865645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:22:03.866463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:03.866463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:22:03.866463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:03.866463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:22:03.866463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:22:03.866463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:03.866463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:22:03.866463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.866463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:22:03.868040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.868040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:03.868040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:03.868040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:03.868040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.868040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:22:03.868040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:22:03.868040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:22:03.868040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:03.868040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:03.868040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:03.868040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:03.868652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:03.868652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:03.868652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:03.868652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:03.868652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:03.868652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:03.868652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:03.868652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:03.868652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:03.868652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:03.868652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:03.868652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:03.868652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:03.868652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:03.869280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:03.869280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:03.869280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.869280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:03.869863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.869863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:03.869863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:03.869863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:03.869863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:03.869863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:03.869863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.869863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:03.869863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:03.869863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.869863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:03.869863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:03.869863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.869863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:03.869863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:03.869863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.870661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:22:03.870661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:22:03.870661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:22:03.870661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:03.870661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:03.870661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:03.870661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:03.870661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:03.870661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:03.870661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:03.870661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:03.870661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:03.870661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:03.870661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:03.870661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:03.870661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:03.870661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:03.870661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:03.871519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:03.871519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:03.871519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:03.871519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:03.871519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:03.871519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.871519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:03.884964 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:03.885374 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:03.885593 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:03.885837 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:03.886376 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:03.887206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.887971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:03.887971 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:03.887971 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:03.887971 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:03.887971 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:03.887971 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:03.887971 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.887971 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:03.889170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.889170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:22:03.889170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:03.889170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:03.889170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:03.889170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:03.889170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:03.889170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:03.889170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:03.889170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:03.889170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:03.889170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:22:03.889170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:03.889170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:22:03.889170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:22:03.890010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:03.890010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:22:03.890010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:03.890010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:22:03.890010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:22:03.890010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:03.890010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:22:03.890010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.890010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:22:03.890604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.890604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:03.890604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:03.890604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:03.890604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.890604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:22:03.890604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:22:03.890604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:03.890604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:03.890604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:03.890604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:03.891173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:03.891173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:03.891173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:03.891173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:03.891173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:03.891173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:03.891173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:03.891173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:03.891173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:03.891173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:03.891173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:03.891173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:03.891173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:03.891173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:03.892026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:03.892026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:03.892026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.892026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:03.892026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.892026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:03.892026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:03.892026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:03.892026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:03.892026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:03.892026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.892026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:03.892026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:03.892026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.892643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:03.892643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:03.892643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.892643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:03.892643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:03.892643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.892643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:22:03.892643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:22:03.892643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:03.892643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:03.892643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:03.892643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:03.892643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:03.892643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:03.892643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:03.892643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:03.892643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:03.892643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:03.892643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:03.893575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:03.893575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:03.893575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:03.893575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:03.893575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:03.893575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:03.893575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:03.893575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:03.893575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:03.893575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.893575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:03.896940 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:03.897251 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:03.897431 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:03.897611 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:03.898098 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:03.898545 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:03.898834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.899641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:03.899641 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:03.899641 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:03.899641 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:03.899641 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:03.899641 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:03.899641 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.899641 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:03.900700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.900700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:03.900700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:03.900700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:03.900700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.900700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:22:03.900700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:03.900700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:03.900700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:03.900700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:03.900700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:03.900700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:03.901134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:03.901134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:03.901134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:03.901134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:03.901134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:03.901134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:03.901134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:03.901134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:03.901134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:03.901134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:03.901134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:03.901134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:03.901134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:03.901474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.901474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:03.901830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.901830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:03.901830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:03.901830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:03.901830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:03.901830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:03.901830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.901830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:03.901830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:03.901830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.901830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:03.901830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:03.901830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.901830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:03.901830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:03.901830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:03.902379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:22:03.902379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:03.902379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:03.902379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:03.902379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:03.902379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:03.902379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:03.902379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:03.902379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:03.902379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:03.902379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:03.902379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:03.902379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:03.902379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:03.902379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:03.902379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:03.902379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:03.902379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:03.902379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:03.903020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:03.903020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:03.903020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:08.568180 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:08.568722 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:08.568822 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:08.569637 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:08.606930 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:08.606930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.607449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:08.607449 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:08.607449 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:08.607449 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:08.607449 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:08.607449 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:08.607449 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.607449 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:08.608252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.608252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:08.608252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:08.608252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:08.608252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.608252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:22:08.608252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:08.608252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:08.608252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:08.608252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:08.608252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:08.608252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:08.608618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:08.608618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:08.608618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:08.608618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:08.608618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:08.608618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:08.608618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:08.608618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:08.608618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:08.608618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:08.608618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:08.608618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:08.608618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:08.608915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.608915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:08.609018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.609018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:08.609018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:08.609018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:08.609018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:08.609018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:08.609018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.609018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:08.609018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:08.609018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.609018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:08.609018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:08.609018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.609018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:08.609018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:08.609018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.609366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:22:08.609366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:08.609366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:08.609366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:08.609366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:08.609366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:08.609366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:08.609366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:08.609366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:08.609366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:08.609366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:08.609366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:08.609366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:08.609366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:08.609366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:08.609366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:08.609366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:08.609366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:08.609366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:08.609802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:08.609802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.609802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:08.802823 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:08.803329 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:08.803455 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:08.804360 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:08.804640 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:08.804867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.806831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:08.806831 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:08.806831 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:08.806831 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:08.806831 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:08.806831 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:08.806831 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.806831 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:08.807364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.807364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:08.807364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:08.807364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:08.807364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.807364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:22:08.807364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:08.807364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:08.807364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:08.807364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:08.807364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:08.807364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:08.807630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:08.807630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:08.807630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:08.807630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:08.807630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:08.807630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:08.807630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:08.807630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:08.807630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:08.807630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:08.807630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:08.807630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:08.807630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:08.808238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.808238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:08.808827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.808827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:08.808827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:08.808827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:08.808827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:08.808827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:08.808827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.808827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:08.808827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:08.808827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.808827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:08.808827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:08.808827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.808827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:08.808827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:08.808827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.809153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:22:08.809153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:08.809153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:08.809153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:08.809153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:08.809153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:08.809153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:08.809153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:08.809153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:08.809153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:08.809153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:08.809153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:08.809153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:08.809153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:08.809153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:08.809153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:08.809153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:08.809153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:08.809153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:08.809500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:08.809500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.809500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:08.861563 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:08.862065 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:08.862269 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:08.862464 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:22:08.863973 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:08.864148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.865054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:08.865054 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:08.865054 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:08.865054 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:08.865054 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:08.865054 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:08.865054 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.865054 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:08.866172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.866172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:22:08.866172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:08.866172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:08.866172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:08.866172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:08.866172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:08.866172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:08.866172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:08.866172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:08.866172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:08.866172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:22:08.866172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:08.866172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:22:08.866172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:22:08.867298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:08.867298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:22:08.867298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:08.867298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:22:08.867298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:22:08.867298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:08.867298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:22:08.867298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.867298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:22:08.867696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.867696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:08.867696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:08.867696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:08.867696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.867696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:22:08.867696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:22:08.867696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:22:08.867696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:08.867696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:08.867696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:08.867696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:08.868168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:08.868168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:08.868168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:08.868168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:08.868168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:08.868168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:08.868168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:08.868168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:08.868168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:08.868168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:08.868168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:08.868168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:08.868168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:08.868168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:08.868711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:08.868711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:08.868711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.868711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:08.869359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.869359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:08.869359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:08.869359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:08.869359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:08.869359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:08.869359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.869359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:08.869359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:08.869359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.869359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:08.869359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:08.869359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.869359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:08.869359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:08.869359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.870009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:22:08.870009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:22:08.870009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:22:08.870009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:08.870009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:08.870009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:08.870009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:08.870009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:08.870009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:08.870009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:08.870009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:08.870009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:08.870009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:08.870009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:08.870009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:08.870009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:08.870009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:08.870009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:08.871057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:08.871057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:08.871057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:08.871057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:08.871057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:08.871057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.871057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:08.885431 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:08.885802 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:08.885985 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:08.886178 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:08.886773 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:08.887060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.888650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:08.888650 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:08.888650 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:08.888650 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:08.888650 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:08.888650 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:08.888650 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.888650 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:08.889852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.889852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:22:08.889852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:08.889852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:08.889852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:08.889852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:08.889852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:08.889852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:08.889852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:08.889852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:08.889852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:08.889852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:22:08.889852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:08.889852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:22:08.889852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:22:08.890251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:08.890251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:22:08.890251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:08.890251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:22:08.890251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:22:08.890251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:08.890251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:22:08.890251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.890251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:22:08.891284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.891284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:08.891284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:08.891284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:08.891284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.891284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:22:08.891284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:22:08.891284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:08.891284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:08.891284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:08.891284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:08.891634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:08.891634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:08.891634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:08.891634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:08.891634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:08.891634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:08.891634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:08.891634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:08.891634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:08.891634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:08.891634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:08.891634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:08.891634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:08.891634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:08.892020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:08.892020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:08.892020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.892020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:08.893251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.893251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:08.893251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:08.893251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:08.893251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:08.893251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:08.893251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.893251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:08.893251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:08.893251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.893251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:08.893251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:08.893251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.893251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:08.893251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:08.893251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.893251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:22:08.894410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:22:08.894410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:08.894410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:08.894410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:08.894410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:08.894410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:08.894410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:08.894410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:08.894410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:08.894410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:08.894410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:08.894410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:08.894410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:08.894410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:08.894410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:08.894410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:08.894410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:08.894410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:08.894410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:08.894914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:08.894914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:08.894914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.894914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:08.897398 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:08.897655 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:08.897783 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:08.897939 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:08.898394 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:08.898605 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:08.898736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.899271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:08.899271 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:08.899271 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:08.899271 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:08.899271 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:08.899271 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:08.899271 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.899271 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:08.900127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.900127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:08.900127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:08.900127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:08.900127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.900127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:22:08.900127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:08.900127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:08.900127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:08.900127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:08.900127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:08.900127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:08.900471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:08.900471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:08.900471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:08.900471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:08.900471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:08.900471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:08.900471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:08.900471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:08.900471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:08.900471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:08.900471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:08.900471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:08.900471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:08.900805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.900805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:08.901099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.901099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:08.901099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:08.901099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:08.901099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:08.901099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:08.901099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.901099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:08.901099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:08.901099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.901099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:08.901099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:08.901099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.901099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:08.901099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:08.901099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:08.902104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:22:08.902104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:08.902104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:08.902104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:08.902104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:08.902104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:08.902104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:08.902104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:08.902104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:08.902104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:08.902104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:08.902104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:08.902104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:08.902104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:08.902104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:08.902104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:08.902104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:08.902104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:08.902104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:08.902598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:08.902598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:08.902598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:13.568776 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:13.569523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:13.569523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:13.570334 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:13.606440 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:13.606631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.607169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:13.607169 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:13.607169 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:13.607169 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:13.607169 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:13.607169 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:13.607169 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.607169 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:13.608026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.608026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:13.608026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:13.608026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:13.608026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.608026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:22:13.608026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:13.608026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:13.608026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:13.608026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:13.608026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:13.608026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:13.608698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:13.608698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:13.608698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:13.608698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:13.608698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:13.608698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:13.608698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:13.608698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:13.608698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:13.608698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:13.608698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:13.608698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:13.608698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:13.609243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.609243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:13.609243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.609243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:13.609243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:13.609243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:13.609243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:13.609243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:13.609243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.609243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:13.609243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:13.609243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.609243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:13.609243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:13.609243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.609922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:13.609922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:13.609922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.609922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:22:13.609922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:13.609922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:13.609922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:13.609922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:13.609922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:13.609922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:13.609922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:13.609922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:13.609922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:13.609922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:13.609922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:13.609922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:13.609922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:13.609922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:13.609922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:13.610724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:13.610724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:13.610724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:13.610724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:13.610724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.610724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:13.803400 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:13.803938 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:13.804102 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:13.804950 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:13.805137 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:13.805252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.805757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:13.805757 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:13.805757 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:13.805757 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:13.805757 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:13.805757 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:13.805757 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.805757 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:13.806825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.806825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:13.806825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:13.806825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:13.806825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.806825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:22:13.806825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:13.806825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:13.806825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:13.806825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:13.806825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:13.806825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:13.807179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:13.807179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:13.807179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:13.807179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:13.807179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:13.807179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:13.807179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:13.807179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:13.807179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:13.807179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:13.807179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:13.807179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:13.807179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:13.807478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.807478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:13.808333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.808333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:13.808333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:13.808333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:13.808333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:13.808333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:13.808333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.808333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:13.808333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:13.808333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.808333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:13.808333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:13.808333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.808333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:13.808333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:13.808333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.808791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:22:13.808791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:13.808791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:13.808791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:13.808791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:13.808791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:13.808791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:13.808791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:13.808791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:13.808791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:13.808791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:13.808791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:13.808791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:13.808791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:13.808791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:13.808791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:13.808791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:13.808791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:13.808791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:13.809360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:13.809360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.809360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:13.862210 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:13.862724 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:13.862919 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:13.863134 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:22:13.863727 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:13.863909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.864494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:13.864494 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:13.864494 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:13.864494 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:13.864494 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:13.864494 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:13.864494 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.864494 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:13.865855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.865855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:22:13.865855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:13.865855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:13.865855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:13.865855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:13.865855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:13.865855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:13.865855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:13.865855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:13.865855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:13.865855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:22:13.865855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:13.865855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:22:13.865855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:22:13.866337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:13.866337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:22:13.866337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:13.866337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:22:13.866337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:22:13.866337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:13.866337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:22:13.866337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.866337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:22:13.867187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.867187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:13.867187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:13.867187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:13.867187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.867187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:22:13.867187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:22:13.867187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:22:13.867187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:13.867187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:13.867187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:13.867187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:13.867494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:13.867494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:13.867494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:13.867494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:13.867494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:13.867494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:13.867494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:13.867494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:13.867494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:13.867494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:13.867494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:13.867494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:13.867494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:13.867494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:13.867932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:13.867932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:13.867932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.867932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:13.868990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.868990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:13.868990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:13.868990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:13.868990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:13.868990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:13.868990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.868990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:13.868990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:13.868990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.868990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:13.868990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:13.868990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.868990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:13.868990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:13.868990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.869532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:22:13.869532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:22:13.869532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:22:13.869532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:13.869532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:13.869532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:13.869532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:13.869532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:13.869532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:13.869532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:13.869532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:13.869532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:13.869532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:13.869532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:13.869532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:13.869532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:13.869532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:13.869532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:13.870254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:13.870254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:13.870254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:13.870254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:13.870254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:13.870254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.870254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:13.886298 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:13.886844 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:13.887156 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:13.887292 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:13.887833 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:13.888072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.888611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:13.888611 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:13.888611 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:13.888611 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:13.888611 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:13.888611 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:13.888611 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.888611 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:13.889535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.889535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:22:13.889535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:13.889535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:13.889535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:13.889535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:13.889535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:13.889535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:13.889535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:13.889535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:13.889535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:13.889535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:22:13.889535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:13.889535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:22:13.889535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:22:13.889879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:13.889879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:22:13.889879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:13.889879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:22:13.889879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:22:13.889879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:13.889879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:22:13.889879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.889879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:22:13.890506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.890506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:13.890506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:13.890506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:13.890506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.890506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:22:13.890506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:22:13.890506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:13.890506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:13.890506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:13.890506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:13.890793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:13.890793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:13.890793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:13.890793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:13.890793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:13.890793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:13.890793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:13.890793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:13.890793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:13.890793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:13.890793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:13.890793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:13.890793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:13.890793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:13.891075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:13.891075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:13.891075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.891075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:13.892703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.892703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:13.892703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:13.892703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:13.892703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:13.892703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:13.892703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.892703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:13.892703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:13.892703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.892703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:13.892703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:13.892703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.892703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:13.892703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:13.892703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.892703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:22:13.893083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:22:13.893083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:13.893083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:13.893083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:13.893083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:13.893083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:13.893083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:13.893083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:13.893083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:13.893083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:13.893083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:13.893083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:13.893083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:13.893083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:13.893083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:13.893083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:13.893083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:13.893083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:13.893083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:13.893460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:13.893460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:13.893460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.893460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:13.897848 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:13.898448 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:13.898601 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:13.898803 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:13.899129 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:13.899437 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:13.899526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.899970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:13.899970 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:13.899970 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:13.899970 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:13.899970 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:13.899970 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:13.899970 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.899970 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:13.900941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.900941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:13.900941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:13.900941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:13.900941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.900941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:22:13.900941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:13.900941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:13.900941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:13.900941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:13.900941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:13.900941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:13.901234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:13.901234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:13.901234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:13.901234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:13.901234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:13.901234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:13.901234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:13.901234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:13.901234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:13.901234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:13.901234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:13.901234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:13.901234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:13.901481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.901481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:13.902030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.902030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:13.902030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:13.902030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:13.902030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:13.902030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:13.902030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.902030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:13.902030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:13.902030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.902030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:13.902030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:13.902030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.902030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:13.902030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:13.902030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:13.902547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:22:13.902547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:13.902547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:13.902547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:13.902547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:13.902547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:13.902547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:13.902547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:13.902547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:13.902547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:13.902547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:13.902547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:13.902547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:13.902547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:13.902547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:13.902547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:13.902547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:13.902547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:13.902547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:13.902972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:13.902972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:13.902972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:18.569403 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:18.570429 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:18.570546 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:18.571361 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:18.605550 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:18.605746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.606291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:18.606291 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:18.606291 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:18.606291 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:18.606291 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:18.606291 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:18.606291 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.606291 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:18.607074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.607074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:18.607074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:18.607074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:18.607074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.607074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:22:18.607074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:18.607074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:18.607074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:18.607074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:18.607074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:18.607074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:18.607482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:18.607482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:18.607482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:18.607482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:18.607482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:18.607482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:18.607482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:18.607482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:18.607482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:18.607482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:18.607482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:18.607482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:18.607482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:18.608128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.608128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:18.608128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.608128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:18.608128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:18.608128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:18.608128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:18.608128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:18.608128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.608128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:18.608128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:18.608128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.608128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:18.608128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:18.608128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.608798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:18.608798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:18.608798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.608798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:22:18.608798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:18.608798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:18.608798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:18.608798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:18.608798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:18.608798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:18.608798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:18.608798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:18.608798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:18.608798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:18.608798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:18.608798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:18.608798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:18.608798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:18.608798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:18.609448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:18.609448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:18.609448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:18.609448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:18.609448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.609448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:18.804266 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:18.804547 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:18.804669 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:18.805571 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:18.805834 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:18.805992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.806639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:18.806639 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:18.806639 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:18.806639 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:18.806639 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:18.806639 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:18.806639 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.806639 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:18.807576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.807576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:18.807576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:18.807576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:18.807576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.807576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:22:18.807576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:18.807576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:18.807576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:18.807576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:18.807576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:18.807576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:18.807863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:18.807863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:18.807863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:18.807863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:18.807863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:18.807863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:18.807863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:18.807863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:18.807863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:18.807863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:18.807863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:18.807863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:18.807863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:18.808197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.808197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:18.808758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.808758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:18.808758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:18.808758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:18.808758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:18.808758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:18.808758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.808758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:18.808758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:18.808758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.808758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:18.808758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:18.808758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.808758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:18.808758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:18.808758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.809175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:22:18.809175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:18.809175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:18.809175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:18.809175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:18.809175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:18.809175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:18.809175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:18.809175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:18.809175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:18.809175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:18.809175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:18.809175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:18.809175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:18.809175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:18.809175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:18.809175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:18.809175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:18.809175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:18.809560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:18.809560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.809560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:18.862670 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:18.862985 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:18.863081 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:18.863243 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:22:18.863808 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:18.864029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.864757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:18.864757 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:18.864757 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:18.864757 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:18.864757 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:18.864757 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:18.864757 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.864757 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:18.865660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.865660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:22:18.865660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:18.865660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:18.865660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:18.865660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:18.865660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:18.865660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:18.865660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:18.865660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:18.865660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:18.865660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:22:18.865660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:18.865660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:22:18.865660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:22:18.866011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:18.866011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:22:18.866011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:18.866011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:22:18.866011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:22:18.866011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:18.866011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:22:18.866011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.866011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:22:18.867306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.867306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:18.867306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:18.867306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:18.867306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.867306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:22:18.867306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:22:18.867306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:22:18.867306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:18.867306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:18.867306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:18.867306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:18.867700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:18.867700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:18.867700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:18.867700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:18.867700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:18.867700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:18.867700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:18.867700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:18.867700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:18.867700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:18.867700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:18.867700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:18.867700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:18.867700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:18.868521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:18.868521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:18.868521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.868521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:18.868521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.868521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:18.868521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:18.868521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:18.868521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:18.868521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:18.868521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.868521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:18.868521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:18.868521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.869019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:18.869019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:18.869019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.869019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:18.869019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:18.869019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.869019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:22:18.869019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:22:18.869019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:22:18.869019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:18.869019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:18.869019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:18.869019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:18.869019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:18.869019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:18.869019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:18.869019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:18.869019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:18.869019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:18.869486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:18.869486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:18.869486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:18.869486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:18.869486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:18.869486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:18.869486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:18.869486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:18.869486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:18.869486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:18.869486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.869486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:18.886731 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:18.887279 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:18.887428 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:18.887821 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:18.888128 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:18.888356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.889163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:18.889163 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:18.889163 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:18.889163 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:18.889163 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:18.889163 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:18.889163 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.889163 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:18.889936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.889936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:22:18.889936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:18.889936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:18.889936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:18.889936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:18.889936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:18.889936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:18.889936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:18.889936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:18.889936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:18.889936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:22:18.889936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:18.889936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:22:18.889936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:22:18.890262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:18.890262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:22:18.890262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:18.890262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:22:18.890262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:22:18.890262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:18.890262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:22:18.890262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.890262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:22:18.890745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.890745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:18.890745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:18.890745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:18.890745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.890745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:22:18.890745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:22:18.890745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:18.890745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:18.890745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:18.890745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:18.891004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:18.891004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:18.891004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:18.891004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:18.891004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:18.891004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:18.891004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:18.891004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:18.891004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:18.891004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:18.891004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:18.891004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:18.891004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:18.891004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:18.891248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:18.891248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:18.891248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.891248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:18.891334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.891334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:18.891334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:18.891334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:18.891334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:18.891334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:18.891334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.891334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:18.891334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:18.891334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.891334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:18.891334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:18.891334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.891334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:18.891334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:18.891334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.891334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:22:18.891641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:22:18.891641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:18.891641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:18.891641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:18.891641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:18.891641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:18.891641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:18.891641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:18.891641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:18.891641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:18.891641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:18.891641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:18.891641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:18.891641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:18.891641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:18.891641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:18.891641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:18.891641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:18.891641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:18.892236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:18.892236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:18.892236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.892236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:18.898507 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:18.898760 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:18.898818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:18.899094 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:18.899326 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:18.899538 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:18.899647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.900096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:18.900096 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:18.900096 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:18.900096 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:18.900096 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:18.900096 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:18.900096 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.900096 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:18.900741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.900741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:18.900741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:18.900741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:18.900741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.900741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:22:18.900741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:18.900741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:18.900741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:18.900741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:18.900741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:18.900741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:18.901093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:18.901093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:18.901093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:18.901093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:18.901093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:18.901093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:18.901093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:18.901093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:18.901093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:18.901093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:18.901093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:18.901093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:18.901093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:18.901420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.901420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:18.901420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.901420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:18.901420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:18.901420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:18.901420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:18.901420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:18.901420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.901420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:18.901420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:18.901420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.901420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:18.901420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:18.901420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.901808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:18.901808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:18.901808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:18.901808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:22:18.901808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:18.901808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:18.901808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:18.901808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:18.901808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:18.901808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:18.901808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:18.901808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:18.901808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:18.901808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:18.901808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:18.901808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:18.901808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:18.901808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:18.901808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:18.902276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:18.902276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:18.902276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:18.902276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:18.902276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:18.902276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:23.570301 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:23.570948 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:23.570948 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:23.571595 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:23.611494 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:23.611859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.612479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:23.612479 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:23.612479 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:23.612479 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:23.612479 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:23.612479 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:23.612479 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.612479 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:23.613245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.613245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:23.613245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:23.613245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:23.613245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.613245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:22:23.613245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:23.613245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:23.613245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:23.613245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:23.613245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:23.613245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:23.614047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:23.614047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:23.614047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:23.614047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:23.614047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:23.614047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:23.614047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:23.614047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:23.614047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:23.614047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:23.614047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:23.614047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:23.614047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:23.614956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.614956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:23.614956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.614956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:23.614956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:23.614956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:23.614956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:23.614956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:23.614956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.614956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:23.614956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:23.614956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.614956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:23.614956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:23.614956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.615828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:23.615828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:23.615828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.615828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:22:23.615828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:23.615828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:23.615828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:23.615828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:23.615828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:23.615828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:23.615828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:23.615828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:23.615828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:23.615828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:23.615828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:23.615828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:23.615828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:23.615828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:23.615828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:23.617225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:23.617225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:23.617225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:23.617225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:23.617225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.617225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:23.804952 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:23.805380 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:23.805504 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:23.806275 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:23.806523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:23.808114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.808752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:23.808752 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:23.808752 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:23.808752 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:23.808752 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:23.808752 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:23.808752 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.808752 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:23.809792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.809792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:23.809792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:23.809792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:23.809792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.809792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:22:23.809792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:23.809792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:23.809792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:23.809792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:23.809792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:23.809792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:23.810146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:23.810146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:23.810146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:23.810146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:23.810146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:23.810146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:23.810146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:23.810146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:23.810146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:23.810146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:23.810146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:23.810146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:23.810146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:23.810386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.810386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:23.810920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.810920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:23.810920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:23.810920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:23.810920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:23.810920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:23.810920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.810920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:23.810920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:23.810920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.810920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:23.810920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:23.810920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.810920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:23.810920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:23.810920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.811256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:22:23.811256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:23.811256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:23.811256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:23.811256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:23.811256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:23.811256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:23.811256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:23.811256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:23.811256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:23.811256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:23.811256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:23.811256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:23.811256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:23.811256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:23.811256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:23.811256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:23.811256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:23.811256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:23.811586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:23.811586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.811586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:23.867012 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:23.867960 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:23.870714 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:23.870714 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:22:23.870714 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:23.870714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.870714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:23.870714 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:23.870714 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:23.870714 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:23.870714 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:23.870714 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:23.870714 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.870714 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:23.871740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.871740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:22:23.871740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:23.871740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:23.871740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:23.871740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:23.871740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:23.871740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:23.871740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:23.871740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:23.871740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:23.871740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:22:23.871740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:23.871740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:22:23.871740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:22:23.872072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:23.872072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:22:23.872072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:23.872072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:22:23.872072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:22:23.872072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:23.872072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:22:23.872072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.872072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:22:23.872634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.872634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:23.872634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:23.872634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:23.872634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.872634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:22:23.872634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:22:23.872634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:22:23.872634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:23.872634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:23.872634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:23.872634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:23.873143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:23.873143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:23.873143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:23.873143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:23.873143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:23.873143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:23.873143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:23.873143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:23.873143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:23.873143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:23.873143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:23.873143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:23.873143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:23.873143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:23.873410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:23.873410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:23.873410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.873410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:23.873410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.873410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:23.873410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:23.873410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:23.873410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:23.873410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:23.873410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.873410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:23.873410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:23.873410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.873706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:23.873706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:23.873706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.873706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:23.873706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:23.873706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.873706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:22:23.873706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:22:23.873706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:22:23.873706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:23.873706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:23.873706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:23.873706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:23.873706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:23.873706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:23.873706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:23.873706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:23.873706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:23.873706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:23.874078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:23.874078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:23.874078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:23.874078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:23.874078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:23.874078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:23.874078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:23.874078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:23.874078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:23.874078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:23.874078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.874078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:23.887935 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:23.888562 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:23.888739 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:23.888896 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:23.889736 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:23.890083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.890821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:23.890821 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:23.890821 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:23.890821 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:23.890821 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:23.890821 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:23.890821 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.890821 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:23.891709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.891709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:22:23.891709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:23.891709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:23.891709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:23.891709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:23.891709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:23.891709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:23.891709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:23.891709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:23.891709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:23.891709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:22:23.891709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:23.891709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:22:23.891709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:22:23.892003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:23.892003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:22:23.892003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:23.892003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:22:23.892003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:22:23.892003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:23.892003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:22:23.892003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.892003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:22:23.892489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.892489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:23.892489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:23.892489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:23.892489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.892489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:22:23.892489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:22:23.892489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:23.892489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:23.892489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:23.892489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:23.892739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:23.892739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:23.892739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:23.892739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:23.892739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:23.892739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:23.892739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:23.892739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:23.892739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:23.892739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:23.892739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:23.892739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:23.892739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:23.892739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:23.892991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:23.892991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:23.892991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.892991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:23.893313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.893313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:23.893313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:23.893313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:23.893313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:23.893313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:23.893313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.893313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:23.893313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:23.893313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.893313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:23.893313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:23.893313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.893313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:23.893313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:23.893313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.893313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:22:23.893739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:22:23.893739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:23.893739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:23.893739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:23.893739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:23.893739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:23.893739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:23.893739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:23.893739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:23.893739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:23.893739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:23.893739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:23.893739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:23.893739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:23.893739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:23.893739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:23.893739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:23.893739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:23.893739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:23.894126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:23.894126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:23.894126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.894126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:23.898936 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:23.899311 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:23.899426 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:23.899619 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:23.900427 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:23.900755 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:23.900908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.901661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:23.901661 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:23.901661 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:23.901661 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:23.901661 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:23.901661 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:23.901661 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.901661 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:23.902889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.902889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:23.902889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:23.902889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:23.902889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.902889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:22:23.902889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:23.902889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:23.902889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:23.902889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:23.902889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:23.902889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:23.903158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:23.903158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:23.903158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:23.903158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:23.903158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:23.903158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:23.903158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:23.903158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:23.903158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:23.903158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:23.903158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:23.903158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:23.903158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:23.903412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.903412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:23.904198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.904198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:23.904198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:23.904198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:23.904198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:23.904198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:23.904198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.904198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:23.904198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:23.904198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.904198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:23.904198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:23.904198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.904198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:23.904198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:23.904198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:23.904533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:22:23.904533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:23.904533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:23.904533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:23.904533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:23.904533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:23.904533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:23.904533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:23.904533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:23.904533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:23.904533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:23.904533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:23.904533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:23.904533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:23.904533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:23.904533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:23.904533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:23.904533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:23.904533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:23.904965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:23.904965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:23.904965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:28.570964 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:28.572070 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:28.572070 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:28.572465 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:28.611588 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:28.611818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.612419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:28.612419 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:28.612419 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:28.612419 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:28.612419 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:28.612419 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:28.612419 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.612419 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:28.613856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.613856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:28.613856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:28.613856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:28.613856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.613856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:22:28.613856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:28.613856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:28.613856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:28.613856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:28.613856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:28.613856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:28.614350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:28.614350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:28.614350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:28.614350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:28.614350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:28.614350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:28.614350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:28.614350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:28.614350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:28.614350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:28.614350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:28.614350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:28.614350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:28.614642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.614642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:28.615088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.615088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:28.615088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:28.615088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:28.615088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:28.615088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:28.615088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.615088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:28.615088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:28.615088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.615088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:28.615088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:28.615088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.615088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:28.615088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:28.615088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.616458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:22:28.616458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:28.616458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:28.616458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:28.616458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:28.616458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:28.616458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:28.616458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:28.616458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:28.616458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:28.616458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:28.616458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:28.616458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:28.616458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:28.616458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:28.616458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:28.616458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:28.616458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:28.616458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:28.617157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:28.617157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.617157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:28.805446 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:28.806163 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:28.806359 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:28.807454 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:28.807696 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:28.807856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.808443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:28.808443 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:28.808443 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:28.808443 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:28.808443 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:28.808443 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:28.808443 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.808443 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:28.809493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.809493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:28.809493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:28.809493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:28.809493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.809493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:22:28.809493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:28.809493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:28.809493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:28.809493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:28.809493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:28.809493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:28.810017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:28.810017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:28.810017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:28.810017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:28.810017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:28.810017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:28.810017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:28.810017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:28.810017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:28.810017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:28.810017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:28.810017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:28.810017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:28.810425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.810425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:28.811061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.811061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:28.811061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:28.811061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:28.811061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:28.811061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:28.811061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.811061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:28.811061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:28.811061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.811061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:28.811061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:28.811061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.811061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:28.811061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:28.811061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.811650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:22:28.811650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:28.811650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:28.811650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:28.811650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:28.811650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:28.811650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:28.811650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:28.811650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:28.811650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:28.811650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:28.811650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:28.811650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:28.811650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:28.811650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:28.811650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:28.811650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:28.811650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:28.811650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:28.813033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:28.813033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.813033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:28.867754 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:28.868498 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:28.868598 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:28.868726 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:22:28.870106 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:28.870352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.871067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:28.871067 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:28.871067 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:28.871067 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:28.871067 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:28.871067 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:28.871067 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.871067 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:28.871812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.871812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:22:28.871812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:28.871812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:28.871812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:28.871812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:28.871812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:28.871812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:28.871812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:28.871812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:28.871812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:28.871812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:22:28.871812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:28.871812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:22:28.871812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:22:28.872471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:28.872471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:22:28.872471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:28.872471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:22:28.872471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:22:28.872471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:28.872471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:22:28.872471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.872471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:22:28.872471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.872471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:28.872471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:28.872471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:28.872471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.873081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:22:28.873081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:22:28.873081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:22:28.873081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:28.873081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:28.873081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:28.873081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:28.873081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:28.873081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:28.873081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:28.873081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:28.873081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:28.873081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:28.873598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:28.873598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:28.873598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:28.873598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:28.873598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:28.873598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:28.873598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:28.873598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:28.873598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:28.873598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:28.873598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.873598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:28.874096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.874096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:28.874096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:28.874096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:28.874096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:28.874096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:28.874096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.874096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:28.874096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:28.874096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.874096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:28.874096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:28.874096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.874096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:28.874096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:28.874096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.876237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:22:28.876237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:22:28.876237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:22:28.876237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:28.876237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:28.876237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:28.876237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:28.876237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:28.876237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:28.876237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:28.876237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:28.876237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:28.876237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:28.876237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:28.876237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:28.876237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:28.876237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:28.876237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:28.877447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:28.877447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:28.877447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:28.877447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:28.877447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:28.877447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.877447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:28.888737 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:28.888737 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:28.888737 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:28.889033 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:28.889576 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:28.889958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.890621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:28.890621 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:28.890621 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:28.890621 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:28.890621 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:28.890621 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:28.890621 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.890621 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:28.892035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.892035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:22:28.892035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:28.892035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:28.892035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:28.892035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:28.892035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:28.892035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:28.892035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:28.892035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:28.892035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:28.892035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:22:28.892035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:28.892035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:22:28.892035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:22:28.892380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:28.892380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:22:28.892380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:28.892380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:22:28.892380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:22:28.892380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:28.892380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:22:28.892380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.892380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:22:28.896296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.896296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:28.896296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:28.896296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:28.896296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.896296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:22:28.896296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:22:28.896296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:28.896296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:28.896296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:28.896296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:28.896670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:28.896670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:28.896670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:28.896670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:28.896670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:28.896670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:28.896670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:28.896670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:28.896670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:28.896670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:28.896670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:28.896670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:28.896670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:28.896670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:28.897080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:28.897080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:28.897080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.897080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:28.897746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.897746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:28.897746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:28.897746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:28.897746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:28.897746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:28.897746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.897746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:28.897746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:28.897746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.897746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:28.897746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:28.897746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.897746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:28.897746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:28.897746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.897746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:22:28.898251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:22:28.898251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:28.898251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:28.898251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:28.898251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:28.898251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:28.898251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:28.898251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:28.898251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:28.898251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:28.898251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:28.898251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:28.898251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:28.898251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:28.898251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:28.898251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:28.898251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:28.898251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:28.898251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:28.898765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:28.898765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:28.898765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.898765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:28.899285 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:28.899850 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:28.900106 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:28.900421 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:28.901104 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:28.901642 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:28.901987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.903527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:28.903527 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:28.903527 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:28.903527 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:28.903527 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:28.903527 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:28.903527 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.903527 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:28.904588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.904588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:28.904588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:28.904588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:28.904588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.904588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:22:28.904588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:28.904588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:28.904588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:28.904588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:28.904588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:28.904588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:28.905216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:28.905216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:28.905216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:28.905216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:28.905216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:28.905216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:28.905216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:28.905216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:28.905216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:28.905216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:28.905216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:28.905216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:28.905216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:28.905751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.905751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:28.907040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.907040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:28.907040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:28.907040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:28.907040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:28.907040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:28.907040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.907040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:28.907040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:28.907040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.907040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:28.907040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:28.907040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.907040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:28.907040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:28.907040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:28.907642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:22:28.907642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:28.907642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:28.907642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:28.907642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:28.907642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:28.907642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:28.907642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:28.907642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:28.907642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:28.907642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:28.907642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:28.907642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:28.907642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:28.907642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:28.907642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:28.907642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:28.907642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:28.907642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:28.908861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:28.908861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:28.908861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:33.571390 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:33.573286 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:33.573286 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:33.573286 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:33.604956 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:33.605218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.605854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:33.605854 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:33.605854 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:33.605854 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:33.605854 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:33.605854 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:33.605854 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.605854 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:33.607873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.607873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:33.607873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:33.607873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:33.607873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.607873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:22:33.607873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:33.607873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:33.607873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:33.607873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:33.607873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:33.607873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:33.608396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:33.608396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:33.608396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:33.608396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:33.608396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:33.608396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:33.608396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:33.608396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:33.608396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:33.608396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:33.608396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:33.608396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:33.608396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:33.608937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.608937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:33.608937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.608937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:33.608937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:33.608937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:33.608937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:33.608937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:33.608937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.608937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:33.608937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:33.608937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.608937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:33.608937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:33.608937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.609270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:33.609270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:33.609270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.609270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:22:33.609270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:33.609270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:33.609270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:33.609270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:33.609270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:33.609270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:33.609270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:33.609270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:33.609270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:33.609270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:33.609270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:33.609270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:33.609270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:33.609270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:33.609270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:33.609977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:33.609977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:33.609977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:33.609977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:33.609977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.609977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:33.806262 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:33.807562 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:33.807562 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:33.807983 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:33.808286 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:33.808519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.812576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:33.812576 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:33.812576 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:33.812576 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:33.812576 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:33.812576 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:33.812576 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.812576 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:33.812576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.812576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:33.812576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:33.812576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:33.812576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.812576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:22:33.812576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:33.813145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:33.813145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:33.813145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:33.813145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:33.813145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:33.813145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:33.813145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:33.813145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:33.813145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:33.813145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:33.813145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:33.813145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:33.813145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:33.813145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:33.813513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:33.813513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:33.813513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:33.813513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:33.813513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.813513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:33.813513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.813513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:33.813513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:33.813513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:33.813513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:33.813513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:33.813513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.813513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:33.813921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:33.813921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.813921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:33.813921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:33.813921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.813921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:33.813921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:33.813921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.813921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:22:33.813921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:33.813921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:33.813921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:33.813921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:33.813921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:33.813921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:33.813921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:33.813921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:33.813921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:33.813921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:33.813921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:33.815563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:33.815563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:33.815563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:33.815563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:33.815563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:33.815563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:33.815563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:33.815563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:33.815563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.815563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:33.868804 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:33.869921 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:33.869921 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:33.869921 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:22:33.870278 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:33.870493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.873712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:33.873712 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:33.873712 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:33.873712 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:33.873712 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:33.873712 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:33.873712 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.873712 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:33.873712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.873712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:22:33.873712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:33.873712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:33.873712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:33.873712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:33.873712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:33.873712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:33.873712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:33.873712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:33.874521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:33.874521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:22:33.874521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:33.874521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:22:33.874521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:22:33.874521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:33.874521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:22:33.874521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:33.874521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:22:33.874521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:22:33.874521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:33.874521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:22:33.874521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.874521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:22:33.875785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.875785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:33.875785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:33.875785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:33.875785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.875785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:22:33.875785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:22:33.875785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:22:33.875785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:33.875785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:33.875785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:33.875785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:33.878338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:33.878338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:33.878338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:33.878338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:33.878338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:33.878338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:33.878338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:33.878338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:33.878338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:33.878338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:33.878338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:33.878338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:33.878338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:33.878338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:33.879286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:33.879286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:33.879286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.879286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:33.879286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.879286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:33.879286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:33.879286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:33.879286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:33.879286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:33.879286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.879286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:33.879286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:33.879286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.880289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:33.880289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:33.880289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.880289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:33.880289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:33.880289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.880289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:22:33.880289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:22:33.880289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:22:33.880289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:33.880289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:33.880289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:33.880289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:33.880289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:33.880289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:33.880289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:33.880289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:33.880289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:33.880289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:33.881385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:33.881385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:33.881385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:33.881385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:33.881385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:33.881385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:33.881385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:33.881385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:33.881385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:33.881385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:33.881385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.881385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:33.888933 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:33.889856 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:33.890274 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:33.890725 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:33.892941 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:33.893200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.893887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:33.893887 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:33.893887 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:33.893887 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:33.893887 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:33.893887 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:33.893887 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.893887 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:33.897015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.897015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:22:33.897015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:33.897015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:33.897015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:33.897015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:33.897015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:33.897015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:33.897015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:33.897015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:33.897015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:33.897015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:22:33.897015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:33.897015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:22:33.897015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:22:33.897429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:33.897429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:22:33.897429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:33.897429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:22:33.897429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:22:33.897429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:33.897429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:22:33.897429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.897429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:22:33.898200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.898200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:33.898200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:33.898200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:33.898200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.898200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:22:33.898200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:22:33.898200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:33.898200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:33.898200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:33.898200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:33.898534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:33.898534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:33.898534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:33.898534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:33.898534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:33.898534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:33.898534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:33.898534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:33.898534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:33.898534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:33.898534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:33.898534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:33.898534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:33.898534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:33.898887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:33.898887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:33.898887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.898887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:33.900464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.900464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:33.900464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:33.900464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:33.900464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:33.900464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:33.900464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.900464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:33.900464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:33.900464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.900464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:33.900464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:33.900464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.900464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:33.900464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:33.900464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.900464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:22:33.902464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:22:33.902464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:33.902464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:33.902464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:33.902464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:33.902464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:33.902464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:33.902464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:33.902464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:33.902464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:33.902464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:33.902464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:33.902464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:33.902464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:33.902464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:33.902464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:33.902464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:33.902464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:33.902464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:33.906151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:33.906151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:33.906151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.906151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:33.906151 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:33.906151 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:33.906151 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:33.906151 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:33.906576 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:33.906576 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:33.906576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.906576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:33.906576 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:33.906576 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:33.906576 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:33.906576 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:33.906576 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:33.906576 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.906576 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:33.906576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.906894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:33.906894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:33.906894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:33.906894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.906894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:22:33.906894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:33.906894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:33.906894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:33.906894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:33.906894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:33.906894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:33.906894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:33.906894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:33.906894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:33.907232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:33.907232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:33.907232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:33.907232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:33.907232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:33.907232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:33.907232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:33.907232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:33.907232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:33.907232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:33.907232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.907232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:33.907521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.907521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:33.907521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:33.907521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:33.907521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:33.907521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:33.907521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.907521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:33.907521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:33.907521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.907521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:33.907521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:33.907521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.907521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:33.907521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:33.907521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:33.908960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:22:33.908960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:33.908960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:33.908960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:33.908960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:33.908960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:33.908960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:33.908960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:33.908960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:33.908960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:33.908960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:33.908960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:33.908960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:33.908960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:33.908960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:33.908960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:33.908960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:33.908960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:33.908960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:33.910226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:33.910226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:33.910226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:38.572375 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:38.573050 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:38.573050 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:38.573890 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:38.609115 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:38.609331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.609911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:38.609911 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:38.609911 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:38.609911 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:38.609911 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:38.609911 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:38.609911 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.609911 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:38.610648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.610648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:38.610648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:38.610648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:38.610648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.610648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:22:38.610648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:38.610648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:38.610648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:38.610648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:38.610648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:38.610648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:38.611092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:38.611092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:38.611092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:38.611092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:38.611092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:38.611092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:38.611092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:38.611092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:38.611092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:38.611092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:38.611092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:38.611092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:38.611092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:38.611484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.611484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:38.611484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.611484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:38.611484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:38.611484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:38.611484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:38.611484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:38.611484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.611484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:38.611484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:38.611484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.611484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:38.611484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:38.611484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.611985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:38.611985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:38.611985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.611985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:22:38.611985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:38.611985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:38.611985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:38.611985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:38.611985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:38.611985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:38.611985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:38.611985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:38.611985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:38.611985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:38.611985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:38.611985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:38.611985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:38.611985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:38.611985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:38.612560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:38.612560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:38.612560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:38.612560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:38.612560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.612560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:38.808188 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:38.808752 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:38.809013 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:38.810018 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:38.810264 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:38.810429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.811196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:38.811196 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:38.811196 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:38.811196 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:38.811196 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:38.811196 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:38.811196 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.811196 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:38.814127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.814127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:38.814127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:38.814127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:38.814127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.814127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:22:38.814127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:38.814127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:38.814127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:38.814127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:38.814127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:38.814127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:38.814399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:38.814399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:38.814399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:38.814399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:38.814399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:38.814399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:38.814399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:38.814399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:38.814399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:38.814399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:38.814399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:38.814399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:38.814399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:38.814637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.814637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:38.814637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.814637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:38.814637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:38.814637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:38.814637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:38.814637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:38.814637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.814637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:38.814637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:38.814637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.814637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:38.814637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:38.814637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.814942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:38.814942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:38.814942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.814942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:22:38.814942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:38.814942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:38.814942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:38.814942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:38.814942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:38.814942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:38.814942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:38.814942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:38.814942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:38.814942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:38.814942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:38.814942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:38.814942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:38.814942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:38.814942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:38.815276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:38.815276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:38.815276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:38.815276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:38.815276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.815276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:38.869542 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:38.870034 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:38.870264 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:38.870470 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:22:38.871138 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:38.871368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.872054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:38.872054 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:38.872054 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:38.872054 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:38.872054 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:38.872054 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:38.872054 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.872054 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:38.874643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.874643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:22:38.874643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:38.874643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:38.874643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:38.874643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:38.874643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:38.874643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:38.874643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:38.874643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:38.874643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:38.874643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:22:38.874643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:38.874643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:22:38.874643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:22:38.875136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:38.875136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:22:38.875136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:38.875136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:22:38.875136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:22:38.875136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:38.875136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:22:38.875136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.875136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:22:38.875136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.875136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:38.875136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:38.875136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:38.875136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.875556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:22:38.875556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:22:38.875556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:22:38.875556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:38.875556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:38.875556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:38.875556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:38.875556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:38.875556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:38.875556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:38.875556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:38.875556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:38.875556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:38.875935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:38.875935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:38.875935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:38.875935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:38.875935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:38.875935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:38.875935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:38.875935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:38.875935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:38.875935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:38.875935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.875935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:38.876991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.876991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:38.876991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:38.876991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:38.876991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:38.876991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:38.876991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.876991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:38.876991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:38.876991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.876991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:38.876991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:38.876991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.876991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:38.876991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:38.876991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.877771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:22:38.877771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:22:38.877771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:22:38.877771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:38.877771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:38.877771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:38.877771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:38.877771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:38.877771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:38.877771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:38.877771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:38.877771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:38.877771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:38.877771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:38.877771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:38.877771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:38.877771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:38.877771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:38.878276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:38.878276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:38.878276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:38.878276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:38.878276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:38.878276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.878276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:38.889917 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:38.890131 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:38.890209 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:38.890344 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:38.891367 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:38.891577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.892230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:38.892230 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:38.892230 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:38.892230 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:38.892230 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:38.892230 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:38.892230 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.892230 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:38.893604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.893604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:22:38.893604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:38.893604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:38.893604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:38.893604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:38.893604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:38.893604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:38.893604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:38.893604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:38.893604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:38.893604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:22:38.893604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:38.893604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:22:38.893604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:22:38.894283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:38.894283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:22:38.894283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:38.894283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:22:38.894283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:22:38.894283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:38.894283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:22:38.894283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.894283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:22:38.895062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.895062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:38.895062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:38.895062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:38.895062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.895062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:22:38.895062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:22:38.895062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:38.895062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:38.895062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:38.895062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:38.895344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:38.895344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:38.895344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:38.895344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:38.895344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:38.895344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:38.895344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:38.895344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:38.895344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:38.895344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:38.895344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:38.895344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:38.895344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:38.895344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:38.895618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:38.895618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:38.895618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.895618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:38.897010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.897010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:38.897010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:38.897010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:38.897010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:38.897010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:38.897010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.897010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:38.897010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:38.897010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.897010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:38.897010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:38.897010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.897010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:38.897010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:38.897010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.897010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:22:38.897622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:22:38.897622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:38.897622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:38.897622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:38.897622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:38.897622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:38.897622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:38.897622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:38.897622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:38.897622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:38.897622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:38.897622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:38.897622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:38.897622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:38.897622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:38.897622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:38.897622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:38.897622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:38.897622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:38.898106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:38.898106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:38.898106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.898106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:38.901273 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:38.901604 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:38.901826 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:38.902011 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:38.902500 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:38.902790 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:38.902964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.903545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:38.903545 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:38.903545 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:38.903545 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:38.903545 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:38.903545 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:38.903545 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.903545 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:38.904366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.904366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:38.904366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:38.904366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:38.904366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.904366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:22:38.904366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:38.904366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:38.904366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:38.904366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:38.904366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:38.904366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:38.904652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:38.904652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:38.904652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:38.904652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:38.904652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:38.904652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:38.904652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:38.904652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:38.904652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:38.904652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:38.904652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:38.904652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:38.904652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:38.904959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.904959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:38.905394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.905394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:38.905394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:38.905394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:38.905394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:38.905394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:38.905394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.905394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:38.905394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:38.905394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.905394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:38.905394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:38.905394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.905394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:38.905394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:38.905394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:38.905774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:22:38.905774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:38.905774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:38.905774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:38.905774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:38.905774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:38.905774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:38.905774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:38.905774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:38.905774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:38.905774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:38.905774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:38.905774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:38.905774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:38.905774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:38.905774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:38.905774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:38.905774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:38.905774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:38.906206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:38.906206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:38.906206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:43.573073 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:43.574128 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:43.574128 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:43.574567 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:43.607763 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:43.608166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.609076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:43.609076 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:43.609076 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:43.609076 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:43.609076 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:43.609076 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:43.609076 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.609076 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:43.610218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.610218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:43.610218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:43.610218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:43.610218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.610218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:22:43.610218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:43.610218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:43.610218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:43.610218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:43.610218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:43.610218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:43.610997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:43.610997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:43.610997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:43.610997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:43.610997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:43.610997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:43.610997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:43.610997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:43.610997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:43.610997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:43.610997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:43.610997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:43.610997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:43.612082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.612082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:43.612082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.612082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:43.612082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:43.612082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:43.612082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:43.612082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:43.612082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.612082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:43.612082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:43.612082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.612082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:43.612082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:43.612082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.613130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:43.613130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:43.613130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.613130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:22:43.613130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:43.613130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:43.613130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:43.613130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:43.613130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:43.613130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:43.613130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:43.613130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:43.613130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:43.613130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:43.613130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:43.613130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:43.613130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:43.613130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:43.613130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:43.614803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:43.614803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:43.614803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:43.614803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:43.614803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.614803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:43.808866 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:43.809366 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:43.809563 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:43.810784 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:43.811066 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:43.811307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.812270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:43.812270 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:43.812270 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:43.812270 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:43.812270 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:43.812270 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:43.812270 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.812270 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:43.813635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.813635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:43.813635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:43.813635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:43.813635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.813635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:22:43.813635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:43.813635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:43.813635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:43.813635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:43.813635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:43.813635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:43.814014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:43.814014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:43.814014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:43.814014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:43.814014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:43.814014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:43.814014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:43.814014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:43.814014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:43.814014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:43.814014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:43.814014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:43.814014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:43.814382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.814382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:43.814927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.814927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:43.814927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:43.814927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:43.814927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:43.814927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:43.814927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.814927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:43.814927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:43.814927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.814927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:43.814927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:43.814927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.814927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:43.814927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:43.814927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.815351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:22:43.815351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:43.815351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:43.815351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:43.815351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:43.815351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:43.815351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:43.815351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:43.815351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:43.815351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:43.815351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:43.815351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:43.815351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:43.815351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:43.815351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:43.815351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:43.815351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:43.815351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:43.815351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:43.815937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:43.815937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.815937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:43.870166 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:43.870708 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:43.870808 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:43.870994 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:22:43.871406 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:43.871578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.872081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:43.872081 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:43.872081 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:43.872081 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:43.872081 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:43.872081 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:43.872081 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.872081 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:43.873232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.873232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:22:43.873232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:43.873232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:43.873232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:43.873232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:43.873232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:43.873232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:43.873232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:43.873232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:43.873232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:43.873232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:22:43.873232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:43.873232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:22:43.873232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:22:43.873750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:43.873750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:22:43.873750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:43.873750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:22:43.873750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:22:43.873750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:43.873750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:22:43.873750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.873750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:22:43.874656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.874656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:43.874656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:43.874656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:43.874656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.874656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:22:43.874656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:22:43.874656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:22:43.874656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:43.874656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:43.874656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:43.874656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:43.875282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:43.875282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:43.875282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:43.875282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:43.875282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:43.875282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:43.875282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:43.875282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:43.875282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:43.875282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:43.875282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:43.875282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:43.875282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:43.875282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:43.875852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:43.875852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:43.875852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.875852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:43.876435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.876435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:43.876435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:43.876435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:43.876435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:43.876435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:43.876435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.876435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:43.876435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:43.876435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.876435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:43.876435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:43.876435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.876435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:43.876435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:43.876435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.876877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:22:43.876877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:22:43.876877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:22:43.876877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:43.876877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:43.876877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:43.876877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:43.876877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:43.876877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:43.876877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:43.876877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:43.876877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:43.876877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:43.876877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:43.876877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:43.876877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:43.876877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:43.876877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:43.877199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:43.877199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:43.877199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:43.877199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:43.877199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:43.877199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.877199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:43.890728 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:43.890728 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:43.890975 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:43.891171 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:43.891508 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:43.891720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.892220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:43.892220 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:43.892220 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:43.892220 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:43.892220 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:43.892220 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:43.892220 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.892220 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:43.893364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.893364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:22:43.893364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:43.893364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:43.893364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:43.893364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:43.893364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:43.893364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:43.893364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:43.893364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:43.893364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:43.893364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:22:43.893364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:43.893364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:22:43.893364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:22:43.893808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:43.893808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:22:43.893808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:43.893808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:22:43.893808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:22:43.893808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:43.893808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:22:43.893808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.893808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:22:43.894603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.894603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:43.894603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:43.894603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:43.894603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.894603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:22:43.894603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:22:43.894603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:43.894603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:43.894603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:43.894603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:43.894978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:43.894978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:43.894978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:43.894978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:43.894978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:43.894978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:43.894978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:43.894978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:43.894978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:43.894978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:43.894978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:43.894978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:43.894978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:43.894978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:43.896323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:43.896323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:43.896323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.896323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:43.896323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.896323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:43.896323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:43.896323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:43.896323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:43.896323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:43.896323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.896323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:43.896323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:43.896323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.896721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:43.896721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:43.896721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.896721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:43.896721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:43.896721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.896721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:22:43.896721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:22:43.896721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:43.896721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:43.896721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:43.896721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:43.896721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:43.896721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:43.896721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:43.896721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:43.896721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:43.896721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:43.896721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:43.897217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:43.897217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:43.897217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:43.897217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:43.897217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:43.897217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:43.897217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:43.897217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:43.897217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:43.897217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.897217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:43.901700 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:43.902133 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:43.902240 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:43.902389 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:43.902848 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:43.903250 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:43.903492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.904194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:43.904194 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:43.904194 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:43.904194 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:43.904194 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:43.904194 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:43.904194 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.904194 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:43.905162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.905162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:43.905162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:43.905162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:43.905162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.905162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:22:43.905162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:43.905162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:43.905162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:43.905162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:43.905162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:43.905162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:43.905410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:43.905410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:43.905410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:43.905410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:43.905410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:43.905410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:43.905410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:43.905410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:43.905410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:43.905410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:43.905410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:43.905410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:43.905410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:43.905658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.905658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:43.906210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.906210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:43.906210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:43.906210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:43.906210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:43.906210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:43.906210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.906210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:43.906210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:43.906210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.906210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:43.906210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:43.906210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.906210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:43.906210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:43.906210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:43.906961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:22:43.906961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:43.906961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:43.906961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:43.906961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:43.906961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:43.906961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:43.906961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:43.906961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:43.906961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:43.906961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:43.906961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:43.906961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:43.906961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:43.906961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:43.906961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:43.906961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:43.906961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:43.906961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:43.907890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:43.907890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:43.907890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:48.573628 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:48.577227 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:48.577382 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:48.578178 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:48.635195 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:48.635195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.635409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:48.635409 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:48.635409 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:48.635409 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:48.635409 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:48.635409 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:48.635409 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.635409 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:48.636201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.058 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.636201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:48.636201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:48.636201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:48.636201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.636201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:22:48.636201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:48.636201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:48.636201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:48.636201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:48.636201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:48.636201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:48.636630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:48.636630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:48.636630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:48.636630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:48.636630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:48.636630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:48.636630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:48.636630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:48.636630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:48.636630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:48.636630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:48.636630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:48.636630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:48.637042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.637042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:48.637042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.637042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:48.637042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:48.637042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:48.637042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:48.637042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:48.637042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.637042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:48.637042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:48.637042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.637042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:48.637042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:48.637042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.637487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:48.637487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:48.637487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.637487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:22:48.637487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:48.637487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:48.637487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:48.637487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:48.637487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:48.637487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:48.637487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:48.637487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:48.637487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:48.637487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:48.637487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:48.637487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:48.637487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:48.637487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:48.637487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:48.638507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:48.638507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:48.638507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:48.638507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:48.638507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.638507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:48.809518 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:48.809998 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:48.810070 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:48.810907 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:48.811151 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:48.811351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.811933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:48.811933 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:48.811933 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:48.811933 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:48.811933 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:48.811933 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:48.811933 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.811933 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:48.812666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.812666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:48.812666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:48.812666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:48.812666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.812666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:22:48.812666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:48.812666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:48.812666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:48.812666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:48.812666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:48.812666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:48.813060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:48.813060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:48.813060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:48.813060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:48.813060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:48.813060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:48.813060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:48.813060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:48.813060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:48.813060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:48.813060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:48.813060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:48.813060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:48.813444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.813444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:48.814079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.814079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:48.814079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:48.814079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:48.814079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:48.814079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:48.814079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.814079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:48.814079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:48.814079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.814079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:48.814079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:48.814079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.814079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:48.814079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:48.814079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.814889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:22:48.814889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:48.814889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:48.814889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:48.814889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:48.814889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:48.814889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:48.814889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:48.814889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:48.814889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:48.814889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:48.814889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:48.814889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:48.814889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:48.814889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:48.814889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:48.814889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:48.814889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:48.814889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:48.815752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:48.815752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.815752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:48.870694 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:48.871136 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:48.871217 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:48.871344 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:22:48.871881 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:48.872098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.872752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:48.872752 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:48.872752 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:48.872752 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:48.872752 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:48.872752 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:48.872752 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.872752 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:48.873385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.873385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:22:48.873385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:48.873385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:48.873385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:48.873385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:48.873385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:48.873385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:48.873385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:48.873385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:48.873385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:48.873385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:22:48.873385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:48.873385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:22:48.873385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:22:48.873896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:48.873896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:22:48.873896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:48.873896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:22:48.873896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:22:48.873896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:48.873896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:22:48.873896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.873896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:22:48.874168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.874168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:48.874168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:48.874168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:48.874168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.874168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:22:48.874168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:22:48.874168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:22:48.874168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:48.874168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:48.874168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:48.874168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:48.874499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:48.874499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:48.874499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:48.874499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:48.874499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:48.874499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:48.874499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:48.874499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:48.874499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:48.874499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:48.874499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:48.874499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:48.874499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:48.874499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:48.874941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:48.874941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:48.874941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.874941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:48.874941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.874941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:48.874941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:48.874941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:48.874941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:48.874941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:48.874941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.874941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:48.874941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:48.874941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.875265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:48.875265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:48.875265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.875265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:48.875265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:48.875265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.875265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:22:48.875265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:22:48.875265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:22:48.875265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:48.875265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:48.875265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:48.875265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:48.875265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:48.875265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:48.875265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:48.875265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:48.875265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:48.875265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:48.875628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:48.875628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:48.875628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:48.875628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:48.875628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:48.875628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:48.875628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:48.875628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:48.875628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:48.875628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:48.875628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.875628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:48.890838 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:48.891284 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:48.891408 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:48.891519 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:48.892151 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:48.892448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.893286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:48.893286 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:48.893286 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:48.893286 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:48.893286 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:48.893286 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:48.893286 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.893286 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:48.894158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.894158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:22:48.894158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:48.894158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:48.894158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:48.894158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:48.894158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:48.894158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:48.894158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:48.894158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:48.894158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:48.894158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:22:48.894158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:48.894158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:22:48.894158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:22:48.894908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:48.894908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:22:48.894908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:48.894908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:22:48.894908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:22:48.894908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:48.894908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:22:48.894908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.894908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:22:48.895286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.895286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:48.895286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:48.895286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:48.895286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.895286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:22:48.895286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:22:48.895286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:48.895286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:48.895286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:48.895286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:48.895775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:48.895775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:48.895775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:48.895775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:48.895775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:48.895775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:48.895775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:48.895775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:48.895775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:48.895775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:48.895775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:48.895775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:48.895775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:48.895775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:48.896361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:48.896361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:48.896361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.896361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:48.896361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.896361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:48.896361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:48.896361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:48.896361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:48.896361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:48.896361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.896361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:48.896361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:48.896361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.897050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:48.897050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:48.897050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.897050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:48.897050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:48.897050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.897050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:22:48.897050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:22:48.897050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:48.897050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:48.897050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:48.897050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:48.897050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:48.897050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:48.897050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:48.897050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:48.897050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:48.897050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:48.897050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:48.897946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:48.897946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:48.897946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:48.897946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:48.897946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:48.897946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:48.897946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:48.897946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:48.897946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:48.897946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.897946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:48.902278 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:48.902617 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:48.902967 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:48.903075 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:48.903573 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:48.903997 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:48.904290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.905221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:48.905221 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:48.905221 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:48.905221 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:48.905221 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:48.905221 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:48.905221 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.905221 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:48.907238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.907238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:48.907238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:48.907238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:48.907238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.907238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:22:48.907238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:48.907238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:48.907238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:48.907238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:48.907238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:48.907238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:48.907890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:48.907890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:48.907890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:48.907890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:48.907890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:48.907890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:48.907890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:48.907890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:48.907890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:48.907890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:48.907890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:48.907890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:48.907890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:48.908388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.908388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:48.909664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.909664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:48.909664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:48.909664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:48.909664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:48.909664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:48.909664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.909664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:48.909664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:48.909664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.909664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:48.909664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:48.909664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.909664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:48.909664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:48.909664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:48.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:22:48.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:48.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:48.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:48.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:48.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:48.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:48.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:48.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:48.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:48.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:48.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:48.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:48.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:48.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:48.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:48.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:48.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:48.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:48.911243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:48.911243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:48.911243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:53.577242 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:53.578116 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:53.578116 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:53.579229 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:53.614366 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:53.614581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.615174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:53.615174 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:53.615174 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:53.615174 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:53.615174 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:53.615174 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:53.615174 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.615174 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:53.615983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.615983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:53.615983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:53.615983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:53.615983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.615983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:22:53.615983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:53.615983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:53.615983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:53.615983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:53.615983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:53.615983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:53.616870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:53.616870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:53.616870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:53.616870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:53.616870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:53.616870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:53.616870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:53.616870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:53.616870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:53.616870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:53.616870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:53.616870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:53.616870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:53.617602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.617602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:53.617602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.617602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:53.617602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:53.617602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:53.617602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:53.617602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:53.617602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.617602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:53.617602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:53.617602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.617602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:53.617602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:53.617602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.618528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:53.618528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:53.618528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.618528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:22:53.618528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:53.618528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:53.618528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:53.618528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:53.618528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:53.618528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:53.618528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:53.618528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:53.618528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:53.618528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:53.618528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:53.618528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:53.618528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:53.618528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:53.618528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:53.619998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:53.619998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:53.619998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:53.619998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:53.619998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.619998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:53.810140 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:53.810704 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:53.810925 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:53.811774 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:53.811891 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:53.812033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.812638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:53.812638 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:53.812638 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:53.812638 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:53.812638 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:53.812638 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:53.812638 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.812638 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:53.814188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.814188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:53.814188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:53.814188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:53.814188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.814188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:22:53.814188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:53.814188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:53.814188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:53.814188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:53.814188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:53.814188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:53.814870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:53.814870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:53.814870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:53.814870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:53.814870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:53.814870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:53.814870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:53.814870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:53.814870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:53.814870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:53.814870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:53.814870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:53.814870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:53.815452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.815452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:53.815926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.815926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:53.815926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:53.815926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:53.815926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:53.815926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:53.815926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.815926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:53.815926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:53.815926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.815926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:53.815926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:53.815926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.815926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:53.815926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:53.815926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.816880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:22:53.816880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:53.816880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:53.816880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:53.816880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:53.816880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:53.816880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:53.816880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:53.816880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:53.816880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:53.816880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:53.816880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:53.816880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:53.816880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:53.816880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:53.816880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:53.816880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:53.816880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:53.816880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:53.818335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:53.818335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.818335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:53.871271 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:53.872173 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:53.872354 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:53.872419 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:22:53.872785 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:53.872967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.873607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:53.873607 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:53.873607 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:53.873607 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:53.873607 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:53.873607 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:53.873607 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.873607 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:53.874615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.874615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:22:53.874615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:53.874615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:53.874615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:53.874615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:53.874615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:53.874615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:53.874615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:53.874615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:53.874615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:53.874615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:22:53.874615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:53.874615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:22:53.874615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:22:53.875511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:53.875511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:22:53.875511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:53.875511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:22:53.875511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:22:53.875511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:53.875511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:22:53.875511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.875511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:22:53.876016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.876016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:53.876016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:53.876016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:53.876016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.876016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:22:53.876016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:22:53.876016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:22:53.876016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:53.876016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:53.876016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:53.876016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:53.876644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:53.876644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:53.876644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:53.876644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:53.876644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:53.876644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:53.876644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:53.876644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:53.876644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:53.876644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:53.876644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:53.876644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:53.876644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:53.876644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:53.877321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:53.877321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:53.877321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.877321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:53.877915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.877915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:53.877915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:53.877915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:53.877915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:53.877915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:53.877915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.877915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:53.877915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:53.877915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.877915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:53.877915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:53.877915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.877915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:53.877915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:53.877915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.878619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:22:53.878619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:22:53.878619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:22:53.878619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:53.878619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:53.878619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:53.878619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:53.878619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:53.878619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:53.878619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:53.878619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:53.878619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:53.878619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:53.878619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:53.878619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:53.878619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:53.878619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:53.878619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:53.879513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:53.879513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:53.879513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:53.879513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:53.879513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:53.879513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.879513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:53.891208 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:53.891537 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:53.891665 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:53.891823 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:53.892205 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:53.892533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.893107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:53.893107 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:53.893107 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:53.893107 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:53.893107 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:53.893107 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:53.893107 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.893107 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:53.894164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.894164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:22:53.894164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:53.894164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:53.894164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:53.894164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:53.894164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:53.894164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:53.894164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:53.894164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:53.894164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:53.894164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:22:53.894164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:53.894164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:22:53.894164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:22:53.894840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:53.894840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:22:53.894840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:53.894840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:22:53.894840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:22:53.894840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:53.894840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:22:53.894840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.894840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:22:53.895774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.895774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:53.895774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:53.895774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:53.895774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.895774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:22:53.895774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:22:53.895774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:53.895774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:53.895774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:53.895774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:53.896408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:53.896408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:53.896408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:53.896408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:53.896408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:53.896408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:53.896408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:53.896408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:53.896408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:53.896408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:53.896408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:53.896408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:53.896408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:53.896408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:53.897026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:53.897026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:53.897026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.897026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:53.897026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.897026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:53.897026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:53.897026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:53.897026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:53.897026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:53.897026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.897026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:53.897026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:53.897026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.897787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:53.897787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:53.897787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.897787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:53.897787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:53.897787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.897787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:22:53.897787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:22:53.897787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:53.897787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:53.897787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:53.897787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:53.897787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:53.897787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:53.897787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:53.897787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:53.897787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:53.897787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:53.897787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:53.898777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:53.898777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:53.898777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:53.898777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:53.898777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:53.898777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:53.898777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:53.898777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:53.898777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:53.898777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.898777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:53.902820 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:53.903137 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:53.903210 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:53.903405 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:53.903741 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:53.903922 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:53.904060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.904489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:53.904489 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:53.904489 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:53.904489 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:53.904489 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:53.904489 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:53.904489 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.904489 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:53.906617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.906617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:53.906617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:53.906617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:53.906617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.906617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:22:53.906617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:53.906617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:53.906617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:53.906617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:53.906617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:53.906617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:53.907025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:53.907025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:53.907025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:53.907025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:53.907025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:53.907025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:53.907025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:53.907025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:53.907025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:53.907025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:53.907025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:53.907025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:53.907025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:53.907383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.907383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:53.907383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.907383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:53.907383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:53.907383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:53.907383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:53.907383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:53.907383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.907383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:53.907383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:53.907383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.907383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:53.907383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:53.907383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.907942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:53.907942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:53.907942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:53.907942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:22:53.907942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:53.907942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:53.907942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:53.907942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:53.907942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:53.907942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:53.907942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:53.907942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:53.907942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:53.907942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:53.907942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:53.907942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:53.907942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:53.907942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:53.907942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:53.908477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:53.908477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:53.908477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:53.908477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:53.908477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:53.908477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:58.577713 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:58.578487 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:58.578487 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:58.580149 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:58.617626 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:58.619072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.621299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:58.621299 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:58.621299 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:58.621299 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:58.621299 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:58.621299 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:58.621299 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.621299 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:58.621299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.621299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:58.621299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:58.621299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:58.621299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.621299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:22:58.621299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:58.621691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:58.621691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:58.621691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:58.621691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:58.621691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:58.621691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:58.621691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:58.621691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:58.621691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:58.621691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:58.621691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:58.621691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:58.621691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:58.621691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:58.621954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:58.621954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:58.621954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:58.621954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:58.621954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.621954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:58.621954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.621954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:58.621954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:58.621954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:58.621954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:58.621954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:58.621954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.621954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:58.622236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:58.622236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.622236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:58.622236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:58.622236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.622236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:58.622236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:58.622236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.622236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:22:58.622236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:58.622236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:58.622236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:58.622236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:58.622236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:58.622236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:58.622236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:58.622236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:58.622236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:58.622236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:58.622236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:58.622236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:58.622631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:58.622631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:58.622631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:58.622631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:58.622631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:58.622631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:58.622631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:58.622631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.622631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:58.810742 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:58.812168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:58.812168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:58.812793 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:58.813204 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:58.813615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.814566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:58.814566 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:58.814566 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:58.814566 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:58.814566 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:58.814566 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:58.814566 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.814566 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:58.816596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.816596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:58.816596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:58.816596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:58.816596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.816596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:22:58.816596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:58.816596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:58.816596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:58.816596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:58.816596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:58.816596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:58.817302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:58.817302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:58.817302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:58.817302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:58.817302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:58.817302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:58.817302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:58.817302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:58.817302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:58.817302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:58.817302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:58.817302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:58.817302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:58.817756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.817756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:58.819132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.819132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:58.819132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:58.819132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:58.819132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:58.819132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:58.819132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.819132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:58.819132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:58.819132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.819132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:58.819132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:58.819132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.819132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:58.819132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:58.819132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.819610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:22:58.819610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:58.819610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:58.819610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:58.819610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:58.819610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:58.819610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:58.819610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:58.819610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:58.819610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:58.819610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:58.819610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:58.819610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:58.819610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:58.819610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:58.819610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:58.819610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:58.819610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:58.819610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:58.820286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:58.820286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.820286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:58.872287 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:58.873816 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:58.873816 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:58.873816 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:22:58.874536 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:58.875082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.878722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:58.878722 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:58.878722 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:58.878722 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:58.878722 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:58.878722 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:58.878722 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.878722 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:58.878722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.878722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:22:58.878722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:58.878722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:58.878722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:58.878722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:58.878722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:58.878722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:58.878722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:58.878722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:58.879258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:58.879258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:22:58.879258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:58.879258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:22:58.879258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:22:58.879258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:58.879258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:22:58.879258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:58.879258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:22:58.879258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:22:58.879258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:58.879258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:22:58.879258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.879258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:22:58.879258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.879703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:58.879703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:58.879703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:58.879703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.879703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:22:58.879703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:22:58.879703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:22:58.879703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:58.879703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:58.879703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:58.879703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:58.879703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:58.879703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:58.880158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:58.880158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:58.880158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:58.880158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:58.880158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:58.880158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:58.880158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:58.880158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:58.880158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:58.880158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:58.880158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:58.880158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:58.880158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:58.880158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:58.881458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.881458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:58.881458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.881458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:58.881458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:58.881458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:58.881458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:58.881458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:58.881458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.881458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:58.881458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:58.881458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.881458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:58.881458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:58.881458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.881882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:58.881882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:58.881882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.881882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:22:58.881882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:22:58.881882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:22:58.881882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:58.881882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:58.881882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:58.881882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:58.881882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:58.881882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:58.881882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:58.881882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:58.881882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:58.881882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:58.881882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:58.881882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:58.881882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:58.882398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:58.882398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:58.882398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:58.882398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:58.882398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:58.882398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:58.882398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:58.882398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.882398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:58.891588 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:58.892053 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:58.892264 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:58.892360 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:58.893574 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:58.893574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.893574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:58.893574 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:58.893574 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:58.893574 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:58.893574 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:58.893574 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:58.893574 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.893574 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:58.894509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.894509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:22:58.894509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:58.894509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:58.894509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:58.894509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:58.894509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:58.894509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:58.894509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:58.894509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:22:58.894509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:58.894509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:22:58.894509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:58.894509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:22:58.894509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:22:58.894859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:58.894859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:22:58.894859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:58.894859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:22:58.894859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:22:58.894859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:58.894859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:22:58.894859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.894859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:22:58.895618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.895618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:58.895618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:58.895618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:58.895618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.895618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:22:58.895618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:22:58.895618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:58.895618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:58.895618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:58.895618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:58.896151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:58.896151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:58.896151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:58.896151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:58.896151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:58.896151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:58.896151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:58.896151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:58.896151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:58.896151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:58.896151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:58.896151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:58.896151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:58.896151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:58.896872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:58.896872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:58.896872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.896872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:58.896991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.896991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:58.896991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:58.896991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:58.896991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:58.896991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:58.896991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.896991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:58.896991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:58.896991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.896991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:58.896991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:58.896991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.896991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:58.896991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:58.896991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.896991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:22:58.897507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:22:58.897507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:22:58.897507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:22:58.897507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:58.897507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:58.897507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:58.897507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:58.897507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:58.897507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:58.897507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:58.897507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:58.897507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:58.897507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:58.897507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:58.897507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:58.897507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:58.897507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:58.897507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:58.897507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:58.898213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:58.898213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:58.898213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.898213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:22:58.903170 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:22:58.903768 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:22:58.903768 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:22:58.903768 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:22:58.904994 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:58.905389 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:22:58.906067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.906361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:22:58.906361 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:58.906361 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:22:58.906361 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:22:58.906361 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:58.906361 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:22:58.906361 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.906361 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:22:58.907605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.907605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:22:58.907605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:58.907605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:22:58.907605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.907605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:22:58.907605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:58.907605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:58.907605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:58.907605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:58.907605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:58.907605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:58.908106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:22:58.908106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:58.908106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:22:58.908106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:58.908106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:22:58.908106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:22:58.908106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:58.908106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:22:58.908106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:58.908106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:22:58.908106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:22:58.908106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:58.908106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:22:58.908465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.908465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:22:58.909639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.909639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:22:58.909639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:22:58.909639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:22:58.909639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:22:58.909639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:22:58.909639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.909639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:22:58.909639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:22:58.909639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.909639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:22:58.909639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:22:58.909639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.909639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:22:58.909639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:22:58.909639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:22:58.910487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:22:58.910487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:22:58.910487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:58.910487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:58.910487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:22:58.910487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:22:58.910487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:22:58.910487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:22:58.910487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:22:58.910487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:22:58.910487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:22:58.910487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:22:58.910487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:22:58.910487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:22:58.910487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:22:58.910487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:22:58.910487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:22:58.910487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:22:58.910487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:22:58.911237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:22:58.911237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:22:58.911237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:03.578384 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:03.579522 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:03.579522 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:03.579522 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:03.587059 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=242 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:23:03.618167 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:03.618167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.618608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:03.618608 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:03.618608 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:03.618608 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:03.618608 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:03.618608 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:03.618608 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.618608 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:03.619838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.619838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:03.619838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:03.619838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:03.619838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.619838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:23:03.619838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:03.619838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:03.619838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:03.619838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:03.619838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:03.619838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:03.620307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:03.620307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:03.620307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:03.620307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:03.620307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:03.620307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:03.620307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:03.620307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:03.620307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:03.620307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:03.620307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:03.620307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:03.620307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:03.620769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.620769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:03.621056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.621056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:03.621056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:03.621056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:03.621056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:03.621056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:03.621056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.621056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:03.621056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:03.621056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.621056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:03.621056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:03.621056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.621056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:03.621056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:03.621056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.621587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:23:03.621587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:03.621587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:03.621587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:03.621587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:03.621587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:03.621587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:03.621587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:03.621587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:03.621587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:03.621587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:03.621587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:03.621587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:03.621587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:03.621587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:03.621587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:03.621587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:03.621587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:03.621587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:03.622174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:03.622174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.622174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:03.811950 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:03.812214 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:03.812431 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:03.813737 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:03.814343 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:03.814343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.815443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:03.815443 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:03.815443 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:03.815443 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:03.815443 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:03.815443 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:03.815443 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.815443 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:03.816809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.816809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:03.816809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:03.816809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:03.816809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.816809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:23:03.816809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:03.816809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:03.816809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:03.816809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:03.816809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:03.816809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:03.817495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:03.817495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:03.817495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:03.817495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:03.817495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:03.817495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:03.817495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:03.817495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:03.817495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:03.817495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:03.817495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:03.817495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:03.817495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:03.818202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.818202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:03.818713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.818713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:03.818713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:03.818713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:03.818713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:03.818713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:03.818713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.818713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:03.818713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:03.818713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.818713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:03.818713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:03.818713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.818713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:03.818713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:03.818713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.819527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:23:03.819527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:03.819527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:03.819527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:03.819527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:03.819527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:03.819527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:03.819527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:03.819527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:03.819527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:03.819527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:03.819527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:03.819527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:03.819527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:03.819527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:03.819527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:03.819527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:03.819527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:03.819527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:03.820487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:03.820487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.820487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:03.873368 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:03.873594 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:03.873801 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:03.873981 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:23:03.874827 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:03.875162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.876082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:03.876082 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:03.876082 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:03.876082 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:03.876082 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:03.876082 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:03.876082 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.876082 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:03.877295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.877295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:23:03.877295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:03.877295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:03.877295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:03.877295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:03.877295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:03.877295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:03.877295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:03.877295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:03.877295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:03.877295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:23:03.877295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:03.877295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:23:03.877295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:23:03.878041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:03.878041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:23:03.878041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:03.878041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:23:03.878041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:23:03.878041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:03.878041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:23:03.878041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.878041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:23:03.878616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.878616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:03.878616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:03.878616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:03.878616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.878616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:23:03.878616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:23:03.878616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:23:03.878616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:03.878616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:03.878616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:03.878616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:03.879205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:03.879205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:03.879205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:03.879205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:03.879205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:03.879205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:03.879205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:03.879205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:03.879205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:03.879205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:03.879205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:03.879205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:03.879205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:03.879205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:03.879863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:03.879863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:03.879863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.879863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:03.880228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.880228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:03.880228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:03.880228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:03.880228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:03.880228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:03.880228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.880228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:03.880228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:03.880228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.880228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:03.880228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:03.880228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.880228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:03.880228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:03.880228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.880958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:23:03.880958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:23:03.880958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:23:03.880958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:03.880958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:03.880958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:03.880958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:03.880958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:03.880958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:03.880958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:03.880958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:03.880958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:03.880958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:03.880958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:03.880958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:03.880958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:03.880958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:03.880958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:03.881880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:03.881880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:03.881880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:03.881880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:03.881880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:03.881880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.881880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:03.892127 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:03.892474 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:03.892657 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:03.892911 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:03.893689 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:03.894213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.895084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:03.895084 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:03.895084 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:03.895084 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:03.895084 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:03.895084 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:03.895084 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.895084 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:03.896402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.896402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:23:03.896402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:03.896402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:03.896402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:03.896402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:03.896402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:03.896402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:03.896402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:03.896402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:03.896402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:03.896402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:23:03.896402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:03.896402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:23:03.896402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:23:03.896820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:03.896820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:23:03.896820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:03.896820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:23:03.896820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:23:03.896820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:03.896820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:23:03.896820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.896820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:23:03.898032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.898032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:03.898032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:03.898032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:03.898032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.898032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:23:03.898032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:23:03.898032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:03.898032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:03.898032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:03.898032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:03.898375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:03.898375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:03.898375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:03.898375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:03.898375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:03.898375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:03.898375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:03.898375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:03.898375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:03.898375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:03.898375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:03.898375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:03.898375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:03.898375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:03.898937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:03.898937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:03.898937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.898937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:03.899758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.899758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:03.899758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:03.899758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:03.899758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:03.899758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:03.899758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.899758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:03.899758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:03.899758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.899758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:03.899758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:03.899758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.899758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:03.899758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:03.899758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.899758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:23:03.900522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:23:03.900522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:03.900522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:03.900522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:03.900522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:03.900522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:03.900522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:03.900522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:03.900522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:03.900522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:03.900522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:03.900522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:03.900522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:03.900522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:03.900522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:03.900522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:03.900522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:03.900522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:03.900522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:03.901338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:03.901338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:03.901338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.901338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:03.903496 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:03.903731 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:03.903834 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:03.904000 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:03.904452 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:03.904888 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:03.905187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.906107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:03.906107 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:03.906107 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:03.906107 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:03.906107 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:03.906107 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:03.906107 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.906107 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:03.906939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.906939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:03.906939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:03.906939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:03.906939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.906939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:23:03.906939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:03.906939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:03.906939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:03.906939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:03.906939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:03.906939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:03.907274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:03.907274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:03.907274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:03.907274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:03.907274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:03.907274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:03.907274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:03.907274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:03.907274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:03.907274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:03.907274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:03.907274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:03.907274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:03.907624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.907624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:03.907889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.907889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:03.907889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:03.907889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:03.907889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:03.907889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:03.907889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.907889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:03.907889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:03.907889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.907889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:03.907889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:03.907889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.907889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:03.907889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:03.907889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:03.908324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:23:03.908324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:03.908324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:03.908324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:03.908324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:03.908324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:03.908324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:03.908324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:03.908324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:03.908324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:03.908324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:03.908324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:03.908324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:03.908324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:03.908324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:03.908324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:03.908324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:03.908324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:03.908324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:03.908855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:03.908855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:03.908855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:08.578947 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:08.579528 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:08.579719 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:08.580905 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:08.622015 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:08.622015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.622015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:08.622015 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:08.622015 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:08.622015 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:08.622015 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:08.622015 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:08.622015 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.622015 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:08.623235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.623235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:08.623235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:08.623235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:08.623235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.623235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:23:08.623235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:08.623235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:08.623235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:08.623235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:08.623235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:08.623235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:08.623669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:08.623669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:08.623669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:08.623669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:08.623669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:08.623669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:08.623669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:08.623669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:08.623669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:08.623669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:08.623669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:08.623669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:08.623669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:08.624236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.624236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:08.625111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.625111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:08.625111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:08.625111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:08.625111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:08.625111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:08.625111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.625111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:08.625111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:08.625111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.625111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:08.625111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:08.625111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.625111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:08.625111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:08.625111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.625792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:23:08.625792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:08.625792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:08.625792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:08.625792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:08.625792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:08.625792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:08.625792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:08.625792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:08.625792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:08.625792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:08.625792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:08.625792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:08.625792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:08.625792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:08.625792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:08.625792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:08.625792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:08.625792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:08.626404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:08.626404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.626404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:08.813135 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:08.814558 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:08.814558 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:08.815006 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:08.815434 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:08.815845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.817002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:08.817002 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:08.817002 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:08.817002 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:08.817002 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:08.817002 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:08.817002 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.817002 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:08.818900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.818900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:08.818900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:08.818900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:08.818900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.818900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:23:08.818900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:08.818900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:08.818900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:08.818900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:08.818900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:08.818900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:08.819299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:08.819299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:08.819299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:08.819299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:08.819299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:08.819299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:08.819299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:08.819299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:08.819299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:08.819299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:08.819299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:08.819299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:08.819299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:08.819696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.819696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:08.820005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.820005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:08.820005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:08.820005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:08.820005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:08.820005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:08.820005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.820005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:08.820005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:08.820005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.820005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:08.820005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:08.820005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.820005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:08.820005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:08.820005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.820513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:23:08.820513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:08.820513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:08.820513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:08.820513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:08.820513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:08.820513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:08.820513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:08.820513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:08.820513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:08.820513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:08.820513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:08.820513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:08.820513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:08.820513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:08.820513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:08.820513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:08.820513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:08.820513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:08.821115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:08.821115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.821115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:08.873872 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:08.874854 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:08.874854 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:08.875133 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:23:08.875662 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:08.876040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.877047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:08.877047 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:08.877047 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:08.877047 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:08.877047 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:08.877047 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:08.877047 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.877047 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:08.877853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.877853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:23:08.877853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:08.877853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:08.877853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:08.877853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:08.877853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:08.877853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:08.877853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:08.877853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:08.877853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:08.877853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:23:08.877853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:08.877853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:23:08.877853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:23:08.878343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:08.878343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:23:08.878343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:08.878343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:23:08.878343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:23:08.878343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:08.878343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:23:08.878343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.878343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:23:08.879428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.879428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:08.879428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:08.879428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:08.879428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.879428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:23:08.879428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:23:08.879428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:23:08.879428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:08.879428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:08.879428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:08.879428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:08.879835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:08.879835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:08.879835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:08.879835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:08.879835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:08.879835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:08.879835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:08.879835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:08.879835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:08.879835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:08.879835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:08.879835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:08.879835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:08.879835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:08.880252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:08.880252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:08.880252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.880252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:08.880252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.880252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:08.880252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:08.880252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:08.880252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:08.880252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:08.880252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.880252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:08.880252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:08.880252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.880660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:08.880660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:08.880660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.880660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:08.880660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:08.880660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.880660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:23:08.880660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:23:08.880660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:23:08.880660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:08.880660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:08.880660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:08.880660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:08.880660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:08.880660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:08.880660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:08.880660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:08.880660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:08.880660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:08.881311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:08.881311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:08.881311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:08.881311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:08.881311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:08.881311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:08.881311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:08.881311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:08.881311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:08.881311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:08.881311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.881311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:08.892706 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:08.893294 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:08.893294 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:08.893458 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:08.894068 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:08.894357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.895787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:08.895787 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:08.895787 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:08.895787 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:08.895787 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:08.895787 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:08.895787 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.895787 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:08.896278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.896278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:23:08.896278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:08.896278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:08.896278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:08.896278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:08.896278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:08.896278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:08.896278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:08.896278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:08.896278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:08.896278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:23:08.896278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:08.896278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:23:08.896278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:23:08.896957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:08.896957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:23:08.896957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:08.896957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:23:08.896957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:23:08.896957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:08.896957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:23:08.896957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.896957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:23:08.896957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.896957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:08.896957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:08.896957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:08.896957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.897755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:23:08.897755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:23:08.897755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:08.897755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:08.897755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:08.897755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:08.897755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:08.897755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:08.897755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:08.897755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:08.897755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:08.897755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:08.897755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:08.898523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:08.898523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:08.898523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:08.898523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:08.898523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:08.898523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:08.898523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:08.898523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:08.898523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:08.898523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.898523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:08.898523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.899146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:08.899146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:08.899146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:08.899146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:08.899146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:08.899146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.899146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:08.899146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:08.899146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.899146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:08.899146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:08.899146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.899146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:08.899146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:08.899146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.899146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:23:08.899146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:23:08.899146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:08.899146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:08.899853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:08.899853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:08.899853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:08.899853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:08.899853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:08.899853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:08.899853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:08.899853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:08.899853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:08.899853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:08.899853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:08.899853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:08.899853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:08.899853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:08.899853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:08.899853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:08.899853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:08.899853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:08.899853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.899853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:08.903948 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:08.904319 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:08.904581 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:08.904789 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:08.905805 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:08.905805 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:08.905805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.906214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:08.906214 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:08.906214 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:08.906214 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:08.906214 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:08.906214 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:08.906214 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.906214 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:08.906979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.906979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:08.906979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:08.906979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:08.906979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.906979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:23:08.906979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:08.906979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:08.906979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:08.906979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:08.906979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:08.906979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:08.907378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:08.907378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:08.907378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:08.907378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:08.907378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:08.907378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:08.907378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:08.907378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:08.907378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:08.907378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:08.907378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:08.907378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:08.907378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:08.907761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.907761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:08.907908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.907908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:08.907908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:08.907908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:08.907908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:08.907908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:08.907908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.907908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:08.907908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:08.907908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.907908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:08.907908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:08.907908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.907908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:08.907908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:08.907908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:08.908360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:23:08.908360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:08.908360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:08.908360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:08.908360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:08.908360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:08.908360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:08.908360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:08.908360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:08.908360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:08.908360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:08.908360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:08.908360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:08.908360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:08.908360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:08.908360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:08.908360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:08.908360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:08.908360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:08.908895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:08.908895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:08.908895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:13.579670 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:13.580296 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:13.580553 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:13.581255 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:13.620790 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:13.620790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.621037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:13.621037 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:13.621037 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:13.621037 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:13.621037 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:13.621037 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:13.621037 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.621037 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:13.622166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.622166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:13.622166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:13.622166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:13.622166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.622166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:23:13.622166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:13.622166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:13.622166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:13.622166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:13.622166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:13.622166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:13.622843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:13.622843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:13.622843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:13.622843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:13.622843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:13.622843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:13.622843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:13.622843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:13.622843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:13.622843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:13.622843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:13.622843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:13.622843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:13.623615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.623615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:13.624276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.624276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:13.624276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:13.624276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:13.624276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:13.624276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:13.624276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.624276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:13.624276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:13.624276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.624276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:13.624276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:13.624276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.624276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:13.624276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:13.624276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.624972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:23:13.624972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:13.624972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:13.624972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:13.624972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:13.624972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:13.624972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:13.624972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:13.624972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:13.624972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:13.624972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:13.624972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:13.624972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:13.624972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:13.624972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:13.624972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:13.624972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:13.624972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:13.624972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:13.625465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:13.625465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.625465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:13.814231 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:13.814777 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:13.815166 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:13.816466 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:13.816818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:13.817020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.818008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:13.818008 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:13.818008 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:13.818008 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:13.818008 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:13.818008 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:13.818008 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.818008 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:13.820142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.820142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:13.820142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:13.820142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:13.820142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.820142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:23:13.820142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:13.820142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:13.820142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:13.820142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:13.820142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:13.820142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:13.820631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:13.820631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:13.820631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:13.820631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:13.820631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:13.820631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:13.820631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:13.820631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:13.820631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:13.820631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:13.820631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:13.820631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:13.820631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:13.821102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.821102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:13.821598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.821598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:13.821598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:13.821598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:13.821598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:13.821598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:13.821598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.821598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:13.821598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:13.821598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.821598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:13.821598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:13.821598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.821598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:13.821598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:13.821598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.822398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:23:13.822398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:13.822398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:13.822398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:13.822398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:13.822398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:13.822398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:13.822398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:13.822398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:13.822398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:13.822398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:13.822398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:13.822398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:13.822398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:13.822398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:13.822398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:13.822398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:13.822398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:13.822398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:13.823041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:13.823041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.823041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:13.874811 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:13.875139 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:13.875470 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:13.875806 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:23:13.876463 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:13.876759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.877472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:13.877472 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:13.877472 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:13.877472 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:13.877472 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:13.877472 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:13.877472 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.877472 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:13.879244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.879244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:23:13.879244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:13.879244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:13.879244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:13.879244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:13.879244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:13.879244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:13.879244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:13.879244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:13.879244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:13.879244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:23:13.879244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:13.879244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:23:13.879244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:23:13.880598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:13.880598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:23:13.880598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:13.880598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:23:13.880598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:23:13.880598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:13.880598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:23:13.880598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.880598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:23:13.882558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.882558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:13.882558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:13.882558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:13.882558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.882558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:23:13.882558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:23:13.882558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:23:13.882558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:13.882558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:13.882558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:13.882558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:13.883189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:13.883189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:13.883189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:13.883189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:13.883189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:13.883189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:13.883189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:13.883189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:13.883189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:13.883189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:13.883189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:13.883189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:13.883189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:13.883189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:13.883849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:13.883849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:13.883849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.883849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:13.884229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.884229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:13.884229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:13.884229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:13.884229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:13.884229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:13.884229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.884229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:13.884229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:13.884229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.884229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:13.884229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:13.884229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.884229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:13.884229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:13.884229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.884597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:23:13.884597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:23:13.884597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:23:13.884597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:13.884597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:13.884597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:13.884597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:13.884597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:13.884597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:13.884597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:13.884597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:13.884597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:13.884597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:13.884597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:13.884597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:13.884597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:13.884597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:13.884597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:13.884985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:13.884985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:13.884985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:13.884985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:13.884985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:13.884985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.884985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:13.893181 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:13.893391 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:13.893549 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:13.893772 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:13.894586 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:13.894955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.895653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:13.895653 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:13.895653 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:13.895653 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:13.895653 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:13.895653 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:13.895653 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.895653 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:13.896697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.896697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:23:13.896697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:13.896697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:13.896697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:13.896697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:13.896697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:13.896697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:13.896697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:13.896697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:13.896697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:13.896697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:23:13.896697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:13.896697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:23:13.896697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:23:13.897347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:13.897347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:23:13.897347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:13.897347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:23:13.897347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:23:13.897347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:13.897347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:23:13.897347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.897347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:23:13.898042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.898042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:13.898042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:13.898042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:13.898042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.898042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:23:13.898042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:23:13.898042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:13.898042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:13.898042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:13.898042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:13.898548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:13.898548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:13.898548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:13.898548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:13.898548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:13.898548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:13.898548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:13.898548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:13.898548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:13.898548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:13.898548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:13.898548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:13.898548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:13.898548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:13.899177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:13.899177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:13.899177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.899177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:13.900504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.900504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:13.900504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:13.900504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:13.900504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:13.900504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:13.900504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.900504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:13.900504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:13.900504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.900504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:13.900504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:13.900504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.900504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:13.900504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:13.900504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.900504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:23:13.901244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:23:13.901244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:13.901244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:13.901244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:13.901244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:13.901244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:13.901244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:13.901244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:13.901244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:13.901244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:13.901244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:13.901244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:13.901244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:13.901244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:13.901244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:13.901244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:13.901244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:13.901244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:13.901244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:13.901943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:13.901943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:13.901943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.901943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:13.904357 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:13.904583 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:13.904757 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:13.904958 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:13.905583 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:13.905913 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:13.906051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.906700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:13.906700 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:13.906700 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:13.906700 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:13.906700 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:13.906700 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:13.906700 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.906700 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:13.907812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.907812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:13.907812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:13.907812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:13.907812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.907812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:23:13.907812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:13.907812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:13.907812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:13.907812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:13.907812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:13.907812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:13.908369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:13.908369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:13.908369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:13.908369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:13.908369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:13.908369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:13.908369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:13.908369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:13.908369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:13.908369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:13.908369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:13.908369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:13.908369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:13.908967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.908967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:13.909455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.909455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:13.909455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:13.909455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:13.909455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:13.909455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:13.909455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.909455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:13.909455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:13.909455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.909455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:13.909455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:13.909455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.909455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:13.909455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:13.909455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:13.910100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:23:13.910100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:13.910100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:13.910100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:13.910100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:13.910100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:13.910100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:13.910100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:13.910100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:13.910100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:13.910100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:13.910100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:13.910100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:13.910100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:13.910100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:13.910100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:13.910100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:13.910100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:13.910100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:13.910731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:13.910731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:13.910731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:18.580547 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:18.581286 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:18.581286 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:18.581823 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:18.617983 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:18.618266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.619034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:18.619034 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:18.619034 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:18.619034 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:18.619034 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:18.619034 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:18.619034 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.619034 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:18.619882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.619882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:18.619882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:18.619882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:18.619882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.619882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:23:18.619882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:18.619882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:18.619882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:18.619882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:18.619882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:18.619882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:18.620532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:18.620532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:18.620532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:18.620532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:18.620532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:18.620532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:18.620532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:18.620532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:18.620532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:18.620532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:18.620532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:18.620532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:18.620532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:18.621205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.621205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:18.621205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.621205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:18.621205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:18.621205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:18.621205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:18.621205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:18.621205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.621205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:18.621205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:18.621205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.621205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:18.621205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:18.621205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.622124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:18.622124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:18.622124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.622124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:23:18.622124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:18.622124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:18.622124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:18.622124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:18.622124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:18.622124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:18.622124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:18.622124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:18.622124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:18.622124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:18.622124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:18.622124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:18.622124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:18.622124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:18.622124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:18.623503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:18.623503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:18.623503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:18.623503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:18.623503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.623503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:18.815023 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:18.815537 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:18.815736 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:18.816890 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:18.817142 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:18.817289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.817964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:18.817964 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:18.817964 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:18.817964 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:18.817964 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:18.817964 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:18.817964 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.817964 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:18.819362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.819362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:18.819362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:18.819362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:18.819362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.819362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:23:18.819362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:18.819362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:18.819362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:18.819362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:18.819362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:18.819362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:18.819949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:18.819949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:18.819949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:18.819949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:18.819949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:18.819949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:18.819949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:18.819949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:18.819949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:18.819949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:18.819949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:18.819949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:18.819949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:18.820451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.820451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:18.820988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.820988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:18.820988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:18.820988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:18.820988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:18.820988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:18.820988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.820988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:18.820988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:18.820988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.820988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:18.820988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:18.820988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.820988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:18.820988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:18.820988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.821597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:23:18.821597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:18.821597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:18.821597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:18.821597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:18.821597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:18.821597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:18.821597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:18.821597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:18.821597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:18.821597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:18.821597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:18.821597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:18.821597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:18.821597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:18.821597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:18.821597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:18.821597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:18.821597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:18.822301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:18.822301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.822301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:18.875279 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:18.875658 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:18.875748 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:18.875849 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:23:18.876425 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:18.876591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.877094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:18.877094 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:18.877094 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:18.877094 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:18.877094 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:18.877094 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:18.877094 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.877094 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:18.877788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.877788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:23:18.877788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:18.877788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:18.877788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:18.877788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:18.877788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:18.877788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:18.877788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:18.877788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:18.877788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:18.877788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:23:18.877788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:18.877788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:23:18.877788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:23:18.878503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:18.878503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:23:18.878503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:18.878503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:23:18.878503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:23:18.878503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:18.878503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:23:18.878503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.878503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:23:18.878503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.878503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:18.878503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:18.878503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:18.878503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.879355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:23:18.879355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:23:18.879355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:23:18.879355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:18.879355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:18.879355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:18.879355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:18.879355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:18.879355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:18.879355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:18.879355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:18.879355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:18.879355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:18.880003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:18.880003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:18.880003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:18.880003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:18.880003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:18.880003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:18.880003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:18.880003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:18.880003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:18.880003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:18.880003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.880003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:18.881149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.881149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:18.881149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:18.881149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:18.881149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:18.881149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:18.881149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.881149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:18.881149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:18.881149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.881149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:18.881149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:18.881149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.881149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:18.881149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:18.881149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.881658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:23:18.881658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:23:18.881658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:23:18.881658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:18.881658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:18.881658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:18.881658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:18.881658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:18.881658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:18.881658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:18.881658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:18.881658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:18.881658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:18.881658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:18.881658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:18.881658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:18.881658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:18.881658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:18.882249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:18.882249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:18.882249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:18.882249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:18.882249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:18.882249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.882249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:18.893690 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:18.894528 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:18.894596 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:18.894856 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:18.895207 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:18.895463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.898217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:18.898217 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:18.898217 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:18.898217 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:18.898217 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:18.898217 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:18.898217 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.898217 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:18.899198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.899198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:23:18.899198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:18.899198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:18.899198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:18.899198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:18.899198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:18.899198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:18.899198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:18.899198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:18.899198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:18.899198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:23:18.899198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:18.899198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:23:18.899198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:23:18.899597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:18.899597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:23:18.899597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:18.899597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:23:18.899597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:23:18.899597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:18.899597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:23:18.899597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.899597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:23:18.900151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.900151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:18.900151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:18.900151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:18.900151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.900151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:23:18.900151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:23:18.900151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:18.900151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:18.900151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:18.900151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:18.900532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:18.900532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:18.900532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:18.900532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:18.900532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:18.900532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:18.900532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:18.900532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:18.900532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:18.900532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:18.900532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:18.900532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:18.900532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:18.900532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:18.901422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:18.901422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:18.901422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.901422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:18.902157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.902157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:18.902157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:18.902157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:18.902157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:18.902157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:18.902157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.902157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:18.902157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:18.902157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.902157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:18.902157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:18.902157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.902157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:18.902157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:18.902157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.902157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:23:18.902733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:23:18.902733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:18.902733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:18.902733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:18.902733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:18.902733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:18.902733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:18.902733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:18.902733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:18.902733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:18.902733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:18.902733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:18.902733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:18.902733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:18.902733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:18.902733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:18.902733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:18.902733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:18.902733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:18.903250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:18.903250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:18.903250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.903250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:18.904775 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:18.905003 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:18.905138 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:18.905330 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:18.905688 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:18.905954 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:18.906197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.906635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:18.906635 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:18.906635 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:18.906635 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:18.906635 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:18.906635 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:18.906635 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.906635 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:18.907611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.907611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:18.907611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:18.907611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:18.907611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.907611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:23:18.907611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:18.907611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:18.907611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:18.907611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:18.907611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:18.907611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:18.908033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:18.908033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:18.908033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:18.908033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:18.908033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:18.908033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:18.908033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:18.908033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:18.908033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:18.908033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:18.908033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:18.908033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:18.908033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:18.908410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.908410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:18.908641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.908641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:18.908641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:18.908641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:18.908641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:18.908641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:18.908641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.908641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:18.908641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:18.908641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.908641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:18.908641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:18.908641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.908641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:18.908641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:18.908641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:18.909308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:23:18.909308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:18.909308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:18.909308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:18.909308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:18.909308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:18.909308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:18.909308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:18.909308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:18.909308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:18.909308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:18.909308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:18.909308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:18.909308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:18.909308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:18.909308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:18.909308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:18.909308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:18.909308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:18.909850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:18.909850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:18.909850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:23.584240 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:23.585257 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:23.585257 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:23.585521 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:23.620010 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:23.620398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.621399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:23.621399 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:23.621399 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:23.621399 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:23.621399 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:23.621399 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:23.621399 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.621399 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:23.622482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.622482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:23.622482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:23.622482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:23.622482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.622482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:23:23.622482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:23.622482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:23.622482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:23.622482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:23.622482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:23.622482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:23.622899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:23.622899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:23.622899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:23.622899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:23.622899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:23.622899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:23.622899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:23.622899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:23.622899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:23.622899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:23.622899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:23.622899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:23.622899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:23.623243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.623243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:23.623905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.623905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:23.623905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:23.623905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:23.623905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:23.623905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:23.623905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.623905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:23.623905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:23.623905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.623905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:23.623905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:23.623905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.623905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:23.623905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:23.623905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.624354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:23:23.624354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:23.624354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:23.624354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:23.624354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:23.624354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:23.624354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:23.624354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:23.624354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:23.624354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:23.624354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:23.624354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:23.624354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:23.624354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:23.624354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:23.624354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:23.624354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:23.624354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:23.624354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:23.624898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:23.624898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.624898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:23.815938 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:23.816692 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:23.816872 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:23.818031 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:23.818280 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:23.818445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.819430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:23.819430 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:23.819430 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:23.819430 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:23.819430 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:23.819430 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:23.819430 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.819430 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:23.820963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.820963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:23.820963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:23.820963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:23.820963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.820963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:23:23.820963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:23.820963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:23.820963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:23.820963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:23.820963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:23.820963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:23.821409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:23.821409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:23.821409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:23.821409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:23.821409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:23.821409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:23.821409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:23.821409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:23.821409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:23.821409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:23.821409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:23.821409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:23.821409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:23.821824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.821824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:23.822417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.822417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:23.822417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:23.822417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:23.822417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:23.822417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:23.822417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.822417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:23.822417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:23.822417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.822417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:23.822417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:23.822417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.822417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:23.822417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:23.822417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.822980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:23:23.822980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:23.822980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:23.822980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:23.822980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:23.822980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:23.822980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:23.822980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:23.822980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:23.822980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:23.822980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:23.822980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:23.822980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:23.822980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:23.822980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:23.822980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:23.822980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:23.822980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:23.822980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:23.823860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:23.823860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.823860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:23.876038 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:23.876710 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:23.876838 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:23.876838 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:23:23.877511 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:23.877755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.878348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:23.878348 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:23.878348 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:23.878348 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:23.878348 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:23.878348 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:23.878348 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.878348 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:23.879011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.879011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:23:23.879011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:23.879011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:23.879011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:23.879011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:23.879011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:23.879011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:23.879011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:23.879011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:23.879011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:23.879011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:23:23.879011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:23.879011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:23:23.879011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:23:23.879930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:23.879930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:23:23.879930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:23.879930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:23:23.879930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:23:23.879930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:23.879930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:23:23.879930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.879930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:23:23.879930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.879930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:23.879930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:23.879930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:23.879930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.880578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:23:23.880578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:23:23.880578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:23:23.880578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:23.880578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:23.880578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:23.880578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:23.880578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:23.880578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:23.880578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:23.880578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:23.880578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:23.880578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:23.881375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:23.881375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:23.881375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:23.881375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:23.881375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:23.881375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:23.881375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:23.881375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:23.881375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:23.881375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:23.881375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.881375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:23.881964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.881964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:23.881964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:23.881964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:23.881964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:23.881964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:23.881964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.881964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:23.881964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:23.881964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.881964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:23.881964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:23.881964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.881964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:23.881964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:23.881964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.883074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:23:23.883074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:23:23.883074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:23:23.883074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:23.883074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:23.883074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:23.883074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:23.883074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:23.883074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:23.883074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:23.883074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:23.883074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:23.883074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:23.883074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:23.883074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:23.883074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:23.883074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:23.883074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:23.884086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:23.884086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:23.884086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:23.884086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:23.884086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:23.884086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.884086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:23.894446 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:23.895058 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:23.895166 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:23.895246 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:23.895835 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:23.896446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.896964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:23.896964 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:23.896964 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:23.896964 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:23.896964 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:23.896964 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:23.896964 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.896964 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:23.897671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.897671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:23:23.897671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:23.897671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:23.897671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:23.897671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:23.897671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:23.897671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:23.897671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:23.897671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:23.897671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:23.897671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:23:23.897671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:23.897671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:23:23.897671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:23:23.898116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:23.898116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:23:23.898116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:23.898116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:23:23.898116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:23:23.898116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:23.898116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:23:23.898116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.898116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:23:23.898449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.898449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:23.898449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:23.898449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:23.898449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.898449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:23:23.898449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:23:23.898449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:23.898449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:23.898449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:23.898449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:23.898752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:23.898752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:23.898752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:23.898752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:23.898752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:23.898752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:23.898752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:23.898752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:23.898752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:23.898752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:23.898752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:23.898752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:23.898752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:23.898752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:23.899092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:23.899092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:23.899092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.899092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:23.900079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.900079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:23.900079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:23.900079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:23.900079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:23.900079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:23.900079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.900079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:23.900079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:23.900079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.900079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:23.900079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:23.900079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.900079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:23.900079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:23.900079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.900079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:23:23.900528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:23:23.900528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:23.900528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:23.900528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:23.900528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:23.900528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:23.900528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:23.900528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:23.900528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:23.900528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:23.900528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:23.900528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:23.900528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:23.900528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:23.900528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:23.900528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:23.900528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:23.900528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:23.900528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:23.900916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:23.900916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:23.900916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.900916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:23.907121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:23.908880 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:23.909009 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:23.909403 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:23.909844 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:23.910183 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:23.910407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.911809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:23.911809 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:23.911809 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:23.911809 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:23.911809 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:23.911809 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:23.911809 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.911809 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:23.912752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.912752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:23.912752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:23.912752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:23.912752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.912752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:23:23.912752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:23.912752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:23.912752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:23.912752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:23.912752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:23.912752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:23.913193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:23.913193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:23.913193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:23.913193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:23.913193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:23.913193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:23.913193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:23.913193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:23.913193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:23.913193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:23.913193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:23.913193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:23.913193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:23.913772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.913772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:23.913772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.913772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:23.913772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:23.913772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:23.913772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:23.913772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:23.913772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.913772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:23.913772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:23.913772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.913772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:23.913772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:23.913772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.914259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:23.914259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:23.914259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:23.914259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:23:23.914259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:23.914259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:23.914259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:23.914259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:23.914259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:23.914259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:23.914259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:23.914259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:23.914259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:23.914259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:23.914259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:23.914259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:23.914259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:23.914259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:23.914259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:23.914981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:23.914981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:23.914981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:23.914981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:23.914981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:23.914981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:28.585088 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:28.587224 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:28.587287 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:28.588080 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:28.623526 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:28.623700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.624691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:28.624691 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:28.624691 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:28.624691 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:28.624691 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:28.624691 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:28.624691 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.624691 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:28.625363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.625363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:28.625363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:28.625363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:28.625363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.625363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:23:28.625363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:28.625363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:28.625363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:28.625363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:28.625363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:28.625363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:28.625771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:28.625771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:28.625771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:28.625771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:28.625771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:28.625771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:28.625771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:28.625771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:28.625771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:28.625771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:28.625771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:28.625771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:28.625771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:28.626747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.626747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:28.626747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.626747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:28.626747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:28.626747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:28.626747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:28.626747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:28.626747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.626747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:28.626747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:28.626747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.626747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:28.626747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:28.626747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.627195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:28.627195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:28.627195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.627195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:23:28.627195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:28.627195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:28.627195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:28.627195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:28.627195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:28.627195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:28.627195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:28.627195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:28.627195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:28.627195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:28.627195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:28.627195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:28.627195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:28.627195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:28.627195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:28.627570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:28.627570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:28.627570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:28.627570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:28.627570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.627570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:28.817341 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:28.817980 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:28.818216 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:28.819291 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:28.819544 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:28.819713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.820494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:28.820494 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:28.820494 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:28.820494 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:28.820494 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:28.820494 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:28.820494 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.820494 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:28.822136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.822136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:28.822136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:28.822136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:28.822136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.822136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:23:28.822136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:28.822136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:28.822136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:28.822136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:28.822136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:28.822136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:28.822553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:28.822553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:28.822553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:28.822553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:28.822553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:28.822553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:28.822553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:28.822553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:28.822553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:28.822553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:28.822553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:28.822553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:28.822553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:28.822941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.822941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:28.824482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.824482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:28.824482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:28.824482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:28.824482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:28.824482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:28.824482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.824482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:28.824482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:28.824482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.824482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:28.824482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:28.824482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.824482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:28.824482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:28.824482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.824851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:23:28.824851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:28.824851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:28.824851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:28.824851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:28.824851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:28.824851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:28.824851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:28.824851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:28.824851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:28.824851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:28.824851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:28.824851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:28.824851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:28.824851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:28.824851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:28.824851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:28.824851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:28.824851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:28.825221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:28.825221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.825221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:28.876654 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:28.877170 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:28.877526 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:28.877839 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:23:28.878615 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:28.878868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.879555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:28.879555 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:28.879555 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:28.879555 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:28.879555 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:28.879555 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:28.879555 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.879555 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:28.880950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.880950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:23:28.880950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:28.880950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:28.880950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:28.880950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:28.880950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:28.880950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:28.880950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:28.880950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:28.880950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:28.880950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:23:28.880950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:28.880950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:23:28.880950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:23:28.881862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:28.881862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:23:28.881862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:28.881862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:23:28.881862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:23:28.881862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:28.881862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:23:28.881862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.881862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:23:28.882408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.882408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:28.882408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:28.882408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:28.882408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.882408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:23:28.882408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:23:28.882408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:23:28.882408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:28.882408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:28.882408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:28.882408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:28.883052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:28.883052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:28.883052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:28.883052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:28.883052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:28.883052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:28.883052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:28.883052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:28.883052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:28.883052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:28.883052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:28.883052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:28.883052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:28.883052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:28.884148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:28.884148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:28.884148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.884148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:28.885516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.885516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:28.885516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:28.885516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:28.885516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:28.885516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:28.885516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.885516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:28.885516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:28.885516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.885516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:28.885516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:28.885516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.885516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:28.885516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:28.885516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.886299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:23:28.886299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:23:28.886299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:23:28.886299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:28.886299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:28.886299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:28.886299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:28.886299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:28.886299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:28.886299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:28.886299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:28.886299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:28.886299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:28.886299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:28.886299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:28.886299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:28.886299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:28.886299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:28.887118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:28.887118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:28.887118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:28.887118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:28.887118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:28.887118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.887118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:28.895175 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:28.895836 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:28.896034 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:28.896242 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:28.897168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:28.897469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.898149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:28.898149 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:28.898149 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:28.898149 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:28.898149 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:28.898149 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:28.898149 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.898149 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:28.899282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.899282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:23:28.899282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:28.899282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:28.899282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:28.899282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:28.899282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:28.899282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:28.899282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:28.899282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:28.899282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:28.899282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:23:28.899282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:28.899282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:23:28.899282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:23:28.899931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:28.899931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:23:28.899931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:28.899931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:23:28.899931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:23:28.899931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:28.899931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:23:28.899931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.899931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:23:28.900967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.900967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:28.900967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:28.900967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:28.900967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.900967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:23:28.900967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:23:28.900967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:28.900967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:28.900967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:28.900967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:28.901461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:28.901461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:28.901461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:28.901461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:28.901461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:28.901461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:28.901461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:28.901461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:28.901461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:28.901461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:28.901461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:28.901461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:28.901461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:28.901461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:28.902065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:28.902065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:28.902065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.902065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:28.902065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.902065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:28.902065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:28.902065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:28.902065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:28.902065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:28.902065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.902065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:28.902065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:28.902065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.903003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:28.903003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:28.903003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.903003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:28.903003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:28.903003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.903003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:23:28.903003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:23:28.903003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:28.903003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:28.903003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:28.903003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:28.903003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:28.903003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:28.903003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:28.903003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:28.903003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:28.903003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:28.903003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:28.903816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:28.903816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:28.903816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:28.903816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:28.903816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:28.903816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:28.903816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:28.903816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:28.903816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:28.903816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.903816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:28.909020 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:28.909359 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:28.909534 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:28.909806 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:28.911373 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:28.911373 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:28.911373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.911373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:28.911373 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:28.911373 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:28.911373 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:28.911373 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:28.911373 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:28.911373 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.911373 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:28.912618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.912618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:28.912618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:28.912618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:28.912618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.912618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:23:28.912618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:28.912618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:28.912618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:28.912618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:28.912618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:28.912618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:28.912903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:28.912903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:28.912903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:28.912903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:28.912903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:28.912903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:28.912903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:28.912903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:28.912903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:28.912903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:28.912903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:28.912903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:28.912903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:28.913149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.913149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:28.915084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.915084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:28.915084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:28.915084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:28.915084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:28.915084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:28.915084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.915084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:28.915084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:28.915084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.915084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:28.915084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:28.915084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.915084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:28.915084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:28.915084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:28.915596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:23:28.915596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:28.915596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:28.915596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:28.915596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:28.915596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:28.915596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:28.915596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:28.915596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:28.915596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:28.915596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:28.915596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:28.915596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:28.915596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:28.915596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:28.915596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:28.915596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:28.915596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:28.915596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:28.916177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:28.916177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:28.916177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:33.587418 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:33.588033 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:33.588252 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:33.589136 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:33.629498 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:33.630800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.630800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:33.630800 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:33.630800 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:33.630800 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:33.630800 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:33.630800 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:33.630800 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.630800 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:33.631635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.631635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:33.631635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:33.631635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:33.631635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.631635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:23:33.631635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:33.631635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:33.631635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:33.631635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:33.631635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:33.631635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:33.633169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:33.633169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:33.633169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:33.633169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:33.633169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:33.633169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:33.633169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:33.633169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:33.633169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:33.633169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:33.633169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:33.633169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:33.633169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:33.634186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.634186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:33.634186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.634186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:33.634186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:33.634186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:33.634186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:33.634186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:33.634186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.634186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:33.634186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:33.634186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.634186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:33.634186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:33.634186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.634900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:33.634900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:33.634900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.634900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:23:33.634900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:33.634900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:33.634900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:33.634900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:33.634900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:33.634900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:33.634900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:33.634900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:33.634900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:33.634900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:33.634900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:33.634900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:33.634900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:33.634900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:33.634900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:33.635755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:33.635755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:33.635755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:33.635755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:33.635755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.635755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:33.817953 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:33.818391 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:33.818510 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:33.819321 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:33.819503 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:33.819627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.820131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:33.820131 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:33.820131 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:33.820131 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:33.820131 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:33.820131 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:33.820131 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.820131 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:33.821286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.821286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:33.821286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:33.821286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:33.821286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.821286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:23:33.821286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:33.821286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:33.821286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:33.821286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:33.821286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:33.821286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:33.821654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:33.821654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:33.821654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:33.821654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:33.821654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:33.821654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:33.821654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:33.821654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:33.821654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:33.821654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:33.821654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:33.821654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:33.821654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:33.822036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.822036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:33.822409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.822409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:33.822409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:33.822409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:33.822409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:33.822409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:33.822409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.822409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:33.822409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:33.822409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.822409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:33.822409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:33.822409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.822409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:33.822409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:33.822409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.822885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:23:33.822885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:33.822885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:33.822885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:33.822885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:33.822885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:33.822885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:33.822885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:33.822885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:33.822885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:33.822885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:33.822885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:33.822885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:33.822885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:33.822885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:33.822885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:33.822885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:33.822885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:33.822885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:33.823465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:33.823465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.823465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:33.877321 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:33.877836 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:33.877836 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:33.877995 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:23:33.878504 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:33.878670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.879249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:33.879249 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:33.879249 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:33.879249 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:33.879249 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:33.879249 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:33.879249 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.879249 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:33.880376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.880376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:23:33.880376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:33.880376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:33.880376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:33.880376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:33.880376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:33.880376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:33.880376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:33.880376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:33.880376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:33.880376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:23:33.880376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:33.880376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:23:33.880376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:23:33.880788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:33.880788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:23:33.880788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:33.880788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:23:33.880788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:23:33.880788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:33.880788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:23:33.880788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.880788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:23:33.881710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.881710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:33.881710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:33.881710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:33.881710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.881710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:23:33.881710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:23:33.881710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:23:33.881710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:33.881710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:33.881710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:33.881710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:33.882010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:33.882010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:33.882010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:33.882010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:33.882010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:33.882010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:33.882010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:33.882010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:33.882010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:33.882010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:33.882010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:33.882010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:33.882010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:33.882010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:33.882256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:33.882256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:33.882256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.882256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:33.883767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.883767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:33.883767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:33.883767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:33.883767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:33.883767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:33.883767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.883767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:33.883767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:33.883767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.883767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:33.883767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:33.883767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.883767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:33.883767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:33.883767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.884325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:23:33.884325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:23:33.884325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:23:33.884325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:33.884325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:33.884325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:33.884325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:33.884325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:33.884325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:33.884325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:33.884325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:33.884325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:33.884325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:33.884325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:33.884325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:33.884325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:33.884325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:33.884325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:33.884968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:33.884968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:33.884968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:33.884968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:33.884968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:33.884968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.884968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:33.895799 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:33.896342 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:33.896634 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:33.896855 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:33.897566 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:33.897793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.898371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:33.898371 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:33.898371 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:33.898371 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:33.898371 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:33.898371 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:33.898371 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.898371 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:33.899476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.899476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:23:33.899476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:33.899476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:33.899476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:33.899476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:33.899476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:33.899476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:33.899476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:33.899476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:33.899476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:33.899476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:23:33.899476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:33.899476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:23:33.899476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:23:33.899806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:33.899806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:23:33.899806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:33.899806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:23:33.899806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:23:33.899806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:33.899806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:23:33.899806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.899806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:23:33.900442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.900442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:33.900442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:33.900442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:33.900442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.900442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:23:33.900442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:23:33.900442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:33.900442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:33.900442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:33.900442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:33.900811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:33.900811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:33.900811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:33.900811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:33.900811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:33.900811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:33.900811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:33.900811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:33.900811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:33.900811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:33.900811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:33.900811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:33.900811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:33.900811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:33.901174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:33.901174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:33.901174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.901174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:33.901656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.901656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:33.901656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:33.901656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:33.901656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:33.901656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:33.901656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.901656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:33.901656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:33.901656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.901656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:33.901656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:33.901656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.901656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:33.901656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:33.901656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.901656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:23:33.902136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:23:33.902136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:33.902136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:33.902136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:33.902136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:33.902136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:33.902136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:33.902136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:33.902136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:33.902136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:33.902136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:33.902136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:33.902136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:33.902136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:33.902136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:33.902136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:33.902136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:33.902136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:33.902136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:33.902623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:33.902623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:33.902623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.902623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:33.910229 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:33.910229 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:33.910417 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:33.910796 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:33.911296 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:33.911490 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:33.911600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.912380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:33.912380 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:33.912380 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:33.912380 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:33.912380 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:33.912380 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:33.912380 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.912380 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:33.913659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.913659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:33.913659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:33.913659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:33.913659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.913659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:23:33.913659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:33.913659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:33.913659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:33.913659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:33.913659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:33.913659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:33.914169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:33.914169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:33.914169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:33.914169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:33.914169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:33.914169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:33.914169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:33.914169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:33.914169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:33.914169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:33.914169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:33.914169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:33.914169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:33.914651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.914651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:33.915671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.915671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:33.915671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:33.915671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:33.915671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:33.915671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:33.915671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.915671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:33.915671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:33.915671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.915671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:33.915671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:33.915671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.915671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:33.915671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:33.915671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:33.916386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:23:33.916386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:33.916386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:33.916386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:33.916386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:33.916386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:33.916386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:33.916386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:33.916386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:33.916386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:33.916386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:33.916386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:33.916386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:33.916386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:33.916386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:33.916386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:33.916386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:33.916386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:33.916386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:33.917074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:33.917074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:33.917074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:38.589719 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:38.589719 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:38.589719 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:38.590406 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:38.623319 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:38.623702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.625617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:38.625617 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:38.625617 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:38.625617 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:38.625617 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:38.625617 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:38.625617 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.625617 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:38.626807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.626807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:38.626807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:38.626807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:38.626807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.626807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:23:38.626807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:38.626807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:38.626807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:38.626807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:38.626807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:38.626807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:38.627297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:38.627297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:38.627297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:38.627297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:38.627297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:38.627297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:38.627297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:38.627297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:38.627297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:38.627297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:38.627297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:38.627297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:38.627297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:38.627598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.627598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:38.627704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.627704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:38.627704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:38.627704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:38.627704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:38.627704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:38.627704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.627704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:38.627704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:38.627704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.627704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:38.627704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:38.627704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.627704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:38.627704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:38.627704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.628224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:23:38.628224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:38.628224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:38.628224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:38.628224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:38.628224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:38.628224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:38.628224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:38.628224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:38.628224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:38.628224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:38.628224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:38.628224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:38.628224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:38.628224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:38.628224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:38.628224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:38.628224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:38.628224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:38.628790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:38.628790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.628790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:38.818475 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:38.819094 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:38.819249 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:38.821090 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:38.821090 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:38.821090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.821260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:38.821260 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:38.821260 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:38.821260 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:38.821260 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:38.821260 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:38.821260 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.821260 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:38.822256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.822256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:38.822256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:38.822256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:38.822256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.822256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:23:38.822256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:38.822256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:38.822256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:38.822256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:38.822256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:38.822256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:38.822649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:38.822649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:38.822649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:38.822649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:38.822649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:38.822649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:38.822649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:38.822649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:38.822649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:38.822649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:38.822649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:38.822649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:38.822649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:38.823036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.823036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:38.823267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.823267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:38.823267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:38.823267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:38.823267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:38.823267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:38.823267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.823267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:38.823267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:38.823267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.823267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:38.823267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:38.823267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.823267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:38.823267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:38.823267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.823822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:23:38.823822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:38.823822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:38.823822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:38.823822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:38.823822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:38.823822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:38.823822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:38.823822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:38.823822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:38.823822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:38.823822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:38.823822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:38.823822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:38.823822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:38.823822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:38.823822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:38.823822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:38.823822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:38.824339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:38.824339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.824339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:38.877874 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:38.878365 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:38.878439 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:38.878579 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:23:38.879098 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:38.880004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.880272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:38.880272 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:38.880272 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:38.880272 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:38.880272 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:38.880272 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:38.880272 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.880272 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:38.881125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.881125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:23:38.881125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:38.881125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:38.881125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:38.881125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:38.881125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:38.881125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:38.881125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:38.881125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:38.881125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:38.881125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:23:38.881125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:38.881125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:23:38.881125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:23:38.881529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:38.881529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:23:38.881529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:38.881529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:23:38.881529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:23:38.881529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:38.881529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:23:38.881529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.881529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:23:38.882005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.882005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:38.882005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:38.882005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:38.882005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.882005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:23:38.882005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:23:38.882005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:23:38.882005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:38.882005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:38.882005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:38.882005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:38.882316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:38.882316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:38.882316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:38.882316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:38.882316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:38.882316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:38.882316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:38.882316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:38.882316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:38.882316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:38.882316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:38.882316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:38.882316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:38.882316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:38.882626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:38.882626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:38.882626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.882626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:38.882981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.882981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:38.882981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:38.882981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:38.882981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:38.882981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:38.882981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.882981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:38.882981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:38.882981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.882981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:38.882981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:38.882981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.882981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:38.882981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:38.882981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.883381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:23:38.883381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:23:38.883381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:23:38.883381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:38.883381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:38.883381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:38.883381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:38.883381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:38.883381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:38.883381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:38.883381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:38.883381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:38.883381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:38.883381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:38.883381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:38.883381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:38.883381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:38.883381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:38.883815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:38.883815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:38.883815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:38.883815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:38.883815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:38.883815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.883815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:38.897147 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:38.897628 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:38.897808 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:38.897944 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:38.898623 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:38.899166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.900209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:38.900209 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:38.900209 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:38.900209 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:38.900209 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:38.900209 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:38.900209 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.900209 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:38.901089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.901089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:23:38.901089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:38.901089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:38.901089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:38.901089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:38.901089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:38.901089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:38.901089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:38.901089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:38.901089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:38.901089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:23:38.901089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:38.901089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:23:38.901089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:23:38.901441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:38.901441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:23:38.901441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:38.901441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:23:38.901441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:23:38.901441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:38.901441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:23:38.901441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.901441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:23:38.902152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.902152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:38.902152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:38.902152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:38.902152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.902152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:23:38.902152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:23:38.902152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:38.902152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:38.902152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:38.902152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:38.902447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:38.902447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:38.902447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:38.902447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:38.902447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:38.902447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:38.902447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:38.902447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:38.902447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:38.902447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:38.902447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:38.902447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:38.902447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:38.902447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:38.903429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:38.903429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:38.903429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.903429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:38.903429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.903429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:38.903429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:38.903429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:38.903429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:38.903429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:38.903429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.903429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:38.903429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:38.903429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.903932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:38.903932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:38.903932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.903932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:38.903932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:38.903932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.903932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:23:38.903932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:23:38.903932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:38.903932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:38.903932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:38.903932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:38.903932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:38.903932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:38.903932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:38.903932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:38.903932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:38.903932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:38.903932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:38.904387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:38.904387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:38.904387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:38.904387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:38.904387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:38.904387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:38.904387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:38.904387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:38.904387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:38.904387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.904387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:38.910049 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:38.910287 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:38.910396 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:38.910558 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:38.911007 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:38.911277 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:38.911450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.912239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:38.912239 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:38.912239 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:38.912239 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:38.912239 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:38.912239 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:38.912239 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.912239 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:38.913691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.913691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:38.913691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:38.913691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:38.913691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.913691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:23:38.913691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:38.913691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:38.913691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:38.913691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:38.913691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:38.913691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:38.914009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:38.914009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:38.914009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:38.914009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:38.914009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:38.914009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:38.914009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:38.914009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:38.914009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:38.914009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:38.914009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:38.914009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:38.914009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:38.914305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.914305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:38.915302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.915302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:38.915302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:38.915302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:38.915302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:38.915302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:38.915302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.915302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:38.915302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:38.915302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.915302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:38.915302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:38.915302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.915302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:38.915302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:38.915302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:38.915895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:23:38.915895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:38.915895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:38.915895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:38.915895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:38.915895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:38.915895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:38.915895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:38.915895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:38.915895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:38.915895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:38.915895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:38.915895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:38.915895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:38.915895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:38.915895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:38.915895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:38.915895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:38.915895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:38.916365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:38.916365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:38.916365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:43.588798 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:43.589365 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:43.589365 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:43.590091 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:43.620907 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:43.621101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.621653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:43.621653 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:43.621653 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:43.621653 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:43.621653 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:43.621653 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:43.621653 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.621653 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:43.622422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.622422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:43.622422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:43.622422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:43.622422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.622422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:23:43.622422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:43.622422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:43.622422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:43.622422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:43.622422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:43.622422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:43.623210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:43.623210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:43.623210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:43.623210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:43.623210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:43.623210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:43.623210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:43.623210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:43.623210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:43.623210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:43.623210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:43.623210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:43.623210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:43.624215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.624215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:43.624215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.624215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:43.624215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:43.624215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:43.624215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:43.624215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:43.624215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.624215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:43.624215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:43.624215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.624215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:43.624215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:43.624215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.625062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:43.625062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:43.625062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.625062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:23:43.625062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:43.625062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:43.625062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:43.625062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:43.625062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:43.625062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:43.625062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:43.625062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:43.625062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:43.625062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:43.625062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:43.625062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:43.625062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:43.625062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:43.625062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:43.625759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:43.625759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:43.625759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:43.625759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:43.625759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.625759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:43.819001 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:43.819310 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:43.819434 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:43.820442 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:43.820850 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:43.821138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.822032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:43.822032 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:43.822032 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:43.822032 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:43.822032 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:43.822032 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:43.822032 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.822032 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:43.823404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.823404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:43.823404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:43.823404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:43.823404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.823404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:23:43.823404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:43.823404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:43.823404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:43.823404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:43.823404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:43.823404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:43.824386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:43.824386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:43.824386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:43.824386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:43.824386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:43.824386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:43.824386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:43.824386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:43.824386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:43.824386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:43.824386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:43.824386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:43.824386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:43.825302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.825302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:43.825302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.825302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:43.825302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:43.825302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:43.825302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:43.825302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:43.825302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.825302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:43.825302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:43.825302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.825302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:43.825302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:43.825302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.826592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:43.826592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:43.826592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.826592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:23:43.826592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:43.826592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:43.826592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:43.826592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:43.826592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:43.826592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:43.826592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:43.826592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:43.826592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:43.826592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:43.826592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:43.826592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:43.826592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:43.826592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:43.826592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:43.827873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:43.827873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:43.827873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:43.827873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:43.827873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.827873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:43.878509 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:43.878876 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:43.878985 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:43.879092 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:23:43.879717 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:43.880053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.880857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:43.880857 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:43.880857 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:43.880857 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:43.880857 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:43.880857 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:43.880857 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.880857 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:43.881648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.881648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:23:43.881648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:43.881648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:43.881648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:43.881648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:43.881648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:43.881648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:43.881648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:43.881648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:43.881648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:43.881648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:23:43.881648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:43.881648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:23:43.881648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:23:43.882387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:43.882387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:23:43.882387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:43.882387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:23:43.882387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:23:43.882387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:43.882387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:23:43.882387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.882387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:23:43.882387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.882387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:43.882387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:43.882387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:43.882387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.883021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:23:43.883021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:23:43.883021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:23:43.883021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:43.883021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:43.883021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:43.883021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:43.883021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:43.883021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:43.883021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:43.883021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:43.883021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:43.883021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:43.883549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:43.883549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:43.883549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:43.883549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:43.883549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:43.883549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:43.883549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:43.883549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:43.883549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:43.883549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:43.883549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.883549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:43.884002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.884002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:43.884002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:43.884002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:43.884002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:43.884002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:43.884002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.884002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:43.884002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:43.884002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.884002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:43.884002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:43.884002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.884002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:43.884002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:43.884002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.884740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:23:43.884740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:23:43.884740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:23:43.884740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:43.884740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:43.884740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:43.884740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:43.884740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:43.884740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:43.884740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:43.884740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:43.884740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:43.884740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:43.884740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:43.884740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:43.884740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:43.884740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:43.884740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:43.885523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:43.885523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:43.885523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:43.885523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:43.885523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:43.885523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.885523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:43.897920 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:43.898158 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:43.898240 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:43.898374 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:43.899006 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:43.899386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.900858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:43.900858 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:43.900858 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:43.900858 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:43.900858 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:43.900858 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:43.900858 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.900858 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:43.901789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.901789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:23:43.901789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:43.901789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:43.901789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:43.901789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:43.901789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:43.901789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:43.901789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:43.901789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:43.901789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:43.901789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:23:43.901789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:43.901789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:23:43.901789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:23:43.902664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:43.902664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:23:43.902664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:43.902664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:23:43.902664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:23:43.902664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:43.902664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:23:43.902664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.902664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:23:43.902664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.902664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:43.902664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:43.902664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:43.902664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.903420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:23:43.903420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:23:43.903420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:43.903420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:43.903420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:43.903420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:43.903420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:43.903420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:43.903420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:43.903420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:43.903420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:43.903420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:43.903420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:43.904016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:43.904016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:43.904016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:43.904016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:43.904016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:43.904016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:43.904016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:43.904016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:43.904016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:43.904016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.904016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:43.904016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.904605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:43.904605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:43.904605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:43.904605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:43.904605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:43.904605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.904605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:43.904605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:43.904605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.904605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:43.904605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:43.904605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.904605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:43.904605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:43.904605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.904605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:23:43.904605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:23:43.904605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:43.904605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:43.905673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:43.905673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:43.905673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:43.905673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:43.905673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:43.905673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:43.905673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:43.905673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:43.905673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:43.905673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:43.905673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:43.905673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:43.905673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:43.905673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:43.905673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:43.905673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:43.905673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:43.905673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:43.905673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.905673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:43.910505 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:43.910780 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:43.910848 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:43.910988 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:43.911544 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:43.911916 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:43.912140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.912838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:43.912838 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:43.912838 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:43.912838 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:43.912838 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:43.912838 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:43.912838 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.912838 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:43.913557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.913557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:43.913557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:43.913557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:43.913557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.913557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:23:43.913557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:43.913557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:43.913557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:43.913557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:43.913557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:43.913557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:43.914118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:43.914118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:43.914118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:43.914118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:43.914118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:43.914118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:43.914118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:43.914118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:43.914118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:43.914118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:43.914118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:43.914118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:43.914118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:43.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:43.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:43.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:43.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:43.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:43.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:43.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:43.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:43.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:43.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:43.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.915231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:43.915231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:43.915231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:43.915231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:23:43.915231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:43.915231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:43.915231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:43.915231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:43.915231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:43.915231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:43.915231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:43.915231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:43.915231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:43.915231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:43.915231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:43.915231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:43.915231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:43.915231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:43.915231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:43.915962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:43.915962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:43.915962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:43.915962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:43.915962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:43.915962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:48.589303 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:48.590198 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:48.590198 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:48.590572 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:48.622293 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:48.622557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.622902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:48.622902 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:48.622902 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:48.622902 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:48.622902 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:48.622902 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:48.622902 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.622902 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:48.623651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.623651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:48.623651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:48.623651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:48.623651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.623651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:23:48.623651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:48.623651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:48.623651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:48.623651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:48.623651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:48.623651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:48.624290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:48.624290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:48.624290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:48.624290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:48.624290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:48.624290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:48.624290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:48.624290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:48.624290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:48.624290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:48.624290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:48.624290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:48.624290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:48.624873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.624873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:48.624873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.624873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:48.624873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:48.624873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:48.624873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:48.624873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:48.624873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.624873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:48.624873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:48.624873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.624873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:48.624873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:48.624873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.625547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:48.625547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:48.625547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.625547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:23:48.625547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:48.625547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:48.625547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:48.625547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:48.625547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:48.625547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:48.625547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:48.625547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:48.625547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:48.625547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:48.625547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:48.625547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:48.625547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:48.625547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:48.625547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:48.628313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:48.628313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:48.628313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:48.628313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:48.628313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.628313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:48.819722 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:48.820021 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:48.820216 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:48.820965 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:48.821084 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:48.821191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.821723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:48.821723 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:48.821723 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:48.821723 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:48.821723 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:48.821723 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:48.821723 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.821723 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:48.823120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.823120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:48.823120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:48.823120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:48.823120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.823120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:23:48.823120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:48.823120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:48.823120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:48.823120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:48.823120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:48.823120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:48.823502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:48.823502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:48.823502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:48.823502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:48.823502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:48.823502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:48.823502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:48.823502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:48.823502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:48.823502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:48.823502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:48.823502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:48.823502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:48.823852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.823852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:48.824904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.824904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:48.824904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:48.824904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:48.824904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:48.824904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:48.824904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.824904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:48.824904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:48.824904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.824904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:48.824904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:48.824904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.824904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:48.824904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:48.824904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.826061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:23:48.826061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:48.826061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:48.826061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:48.826061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:48.826061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:48.826061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:48.826061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:48.826061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:48.826061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:48.826061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:48.826061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:48.826061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:48.826061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:48.826061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:48.826061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:48.826061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:48.826061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:48.826061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:48.826892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:48.826892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.826892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:48.879737 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:48.882215 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:48.882380 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:48.882589 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:23:48.882984 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:48.883131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.883699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:48.883699 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:48.883699 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:48.883699 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:48.883699 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:48.883699 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:48.883699 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.883699 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:48.884857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.884857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:23:48.884857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:48.884857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:48.884857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:48.884857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:48.884857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:48.884857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:48.884857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:48.884857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:48.884857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:48.884857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:23:48.884857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:48.884857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:23:48.884857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:23:48.885449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:48.885449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:23:48.885449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:48.885449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:23:48.885449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:23:48.885449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:48.885449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:23:48.885449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.885449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:23:48.886187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.886187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:48.886187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:48.886187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:48.886187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.886187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:23:48.886187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:23:48.886187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:23:48.886187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:48.886187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:48.886187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:48.886187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:48.886868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:48.886868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:48.886868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:48.886868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:48.886868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:48.886868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:48.886868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:48.886868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:48.886868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:48.886868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:48.886868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:48.886868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:48.886868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:48.886868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:48.887426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:48.887426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:48.887426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.887426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:48.890348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.890348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:48.890348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:48.890348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:48.890348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:48.890348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:48.890348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.890348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:48.890348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:48.890348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.890348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:48.890348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:48.890348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.890348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:48.890348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:48.890348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.891232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:23:48.891232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:23:48.891232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:23:48.891232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:48.891232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:48.891232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:48.891232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:48.891232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:48.891232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:48.891232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:48.891232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:48.891232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:48.891232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:48.891232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:48.891232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:48.891232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:48.891232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:48.891232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:48.892132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:48.892132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:48.892132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:48.892132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:48.892132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:48.892132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.892132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:48.898419 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:48.898800 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:48.898880 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:48.898989 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:48.899365 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:48.899581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.900023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:48.900023 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:48.900023 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:48.900023 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:48.900023 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:48.900023 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:48.900023 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.900023 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:48.900812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.900812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:23:48.900812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:48.900812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:48.900812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:48.900812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:48.900812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:48.900812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:48.900812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:48.900812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:48.900812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:48.900812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:23:48.900812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:48.900812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:23:48.900812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:23:48.901248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:48.901248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:23:48.901248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:48.901248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:23:48.901248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:23:48.901248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:48.901248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:23:48.901248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.901248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:23:48.901731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.901731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:48.901731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:48.901731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:48.901731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.901731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:23:48.901731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:23:48.901731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:48.901731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:48.901731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:48.901731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:48.902101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:48.902101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:48.902101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:48.902101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:48.902101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:48.902101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:48.902101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:48.902101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:48.902101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:48.902101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:48.902101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:48.902101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:48.902101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:48.902101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:48.902477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:48.902477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:48.902477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.902477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:48.902898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.902898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:48.902898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:48.902898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:48.902898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:48.902898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:48.902898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.902898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:48.902898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:48.902898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.902898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:48.902898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:48.902898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.902898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:48.902898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:48.902898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.902898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:23:48.903385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:23:48.903385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:48.903385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:48.903385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:48.903385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:48.903385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:48.903385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:48.903385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:48.903385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:48.903385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:48.903385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:48.903385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:48.903385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:48.903385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:48.903385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:48.903385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:48.903385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:48.903385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:48.903385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:48.903888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:48.903888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:48.903888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.903888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:48.911142 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:48.911529 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:48.911645 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:48.911854 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:48.912152 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:48.912350 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:48.912463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.912905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:48.912905 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:48.912905 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:48.912905 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:48.912905 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:48.912905 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:48.912905 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.912905 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:48.913759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.913759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:48.913759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:48.913759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:48.913759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.913759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:23:48.913759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:48.913759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:48.913759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:48.913759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:48.913759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:48.913759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:48.914607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:48.914607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:48.914607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:48.914607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:48.914607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:48.914607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:48.914607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:48.914607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:48.914607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:48.914607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:48.914607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:48.914607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:48.914607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:48.915224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.915224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:48.915224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.915224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:48.915224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:48.915224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:48.915224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:48.915224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:48.915224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.915224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:48.915224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:48.915224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.915224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:48.915224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:48.915224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.915951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:48.915951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:48.915951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:48.915951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:23:48.915951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:48.915951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:48.915951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:48.915951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:48.915951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:48.915951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:48.915951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:48.915951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:48.915951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:48.915951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:48.915951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:48.915951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:48.915951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:48.915951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:48.915951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:48.917012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:48.917012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:48.917012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:48.917012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:48.917012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:48.917012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:53.589822 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:53.590472 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:53.590472 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:53.591033 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:53.629570 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:53.629745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.630340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:53.630340 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:53.630340 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:53.630340 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:53.630340 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:53.630340 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:53.630340 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.630340 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:53.631168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.631168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:53.631168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:53.631168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:53.631168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.631168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:23:53.631168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:53.631168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:53.631168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:53.631168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:53.631168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:53.631168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:53.631589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:53.631589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:53.631589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:53.631589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:53.631589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:53.631589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:53.631589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:53.631589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:53.631589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:53.631589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:53.631589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:53.631589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:53.631589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:53.632282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.632282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:53.632282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.632282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:53.632282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:53.632282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:53.632282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:53.632282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:53.632282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.632282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:53.632282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:53.632282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.632282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:53.632282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:53.632282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.633012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:53.633012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:53.633012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.633012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:23:53.633012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:53.633012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:53.633012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:53.633012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:53.633012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:53.633012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:53.633012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:53.633012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:53.633012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:53.633012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:53.633012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:53.633012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:53.633012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:53.633012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:53.633012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:53.634105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:53.634105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:53.634105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:53.634105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:53.634105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.634105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:53.820022 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:53.820440 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:53.820664 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:53.824130 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:53.824310 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:53.824418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.824998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:53.824998 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:53.824998 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:53.824998 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:53.824998 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:53.824998 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:53.824998 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.824998 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:53.827570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.827570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:53.827570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:53.827570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:53.827570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.827570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:23:53.827570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:53.827570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:53.827570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:53.827570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:53.827570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:53.827570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:53.827948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:53.827948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:53.827948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:53.827948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:53.827948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:53.827948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:53.827948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:53.827948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:53.827948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:53.827948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:53.827948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:53.827948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:53.827948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:53.828179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.828179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:53.829968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.829968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:53.829968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:53.829968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:53.829968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:53.829968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:53.829968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.829968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:53.829968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:53.829968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.829968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:53.829968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:53.829968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.829968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:53.829968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:53.829968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.830441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:23:53.830441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:53.830441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:53.830441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:53.830441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:53.830441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:53.830441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:53.830441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:53.830441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:53.830441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:53.830441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:53.830441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:53.830441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:53.830441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:53.830441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:53.830441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:53.830441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:53.830441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:53.830441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:53.831009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:53.831009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.831009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:53.882458 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:53.883709 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:53.883709 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:53.883709 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:23:53.883858 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:53.884037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.884652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:53.884652 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:53.884652 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:53.884652 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:53.884652 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:53.884652 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:53.884652 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.884652 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:53.886296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.886296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:23:53.886296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:53.886296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:53.886296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:53.886296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:53.886296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:53.886296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:53.886296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:53.886296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:53.886296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:53.886296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:23:53.886296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:53.886296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:23:53.886296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:23:53.887080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:53.887080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:23:53.887080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:53.887080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:23:53.887080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:23:53.887080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:53.887080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:23:53.887080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.887080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:23:53.888513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.888513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:53.888513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:53.888513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:53.888513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.888513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:23:53.888513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:23:53.888513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:23:53.888513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:53.888513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:53.888513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:53.888513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:53.888980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:53.888980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:53.888980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:53.888980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:53.888980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:53.888980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:53.888980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:53.888980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:53.888980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:53.888980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:53.888980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:53.888980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:53.888980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:53.888980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:53.889569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:53.889569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:53.889569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.889569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:53.889774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.889774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:53.889774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:53.889774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:53.889774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:53.889774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:53.889774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.889774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:53.889774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:53.889774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.889774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:53.889774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:53.889774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.889774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:53.889774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:53.889774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.890219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:23:53.890219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:23:53.890219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:23:53.890219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:53.890219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:53.890219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:53.890219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:53.890219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:53.890219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:53.890219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:53.890219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:53.890219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:53.890219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:53.890219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:53.890219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:53.890219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:53.890219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:53.890219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:53.891316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:53.891316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:53.891316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:53.891316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:53.891316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:53.891316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.891316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:53.898735 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:53.898997 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:53.899133 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:53.899269 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:53.899822 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:53.900030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.900485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:53.900485 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:53.900485 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:53.900485 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:53.900485 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:53.900485 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:53.900485 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.900485 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:53.901241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.901241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:23:53.901241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:53.901241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:53.901241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:53.901241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:53.901241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:53.901241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:53.901241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:53.901241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:53.901241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:53.901241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:23:53.901241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:53.901241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:23:53.901241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:23:53.901744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:53.901744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:23:53.901744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:53.901744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:23:53.901744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:23:53.901744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:53.901744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:23:53.901744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.901744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:23:53.902181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.902181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:53.902181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:53.902181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:53.902181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.902181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:23:53.902181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:23:53.902181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:53.902181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:53.902181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:53.902181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:53.902546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:53.902546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:53.902546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:53.902546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:53.902546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:53.902546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:53.902546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:53.902546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:53.902546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:53.902546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:53.902546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:53.902546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:53.902546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:53.902546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:53.903002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:53.903002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:53.903002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.903002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:53.903461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.903461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:53.903461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:53.903461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:53.903461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:53.903461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:53.903461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.903461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:53.903461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:53.903461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.903461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:53.903461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:53.903461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.903461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:53.903461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:53.903461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.903461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:23:53.904855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:23:53.904855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:53.904855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:53.904855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:53.904855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:53.904855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:53.904855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:53.904855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:53.904855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:53.904855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:53.904855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:53.904855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:53.904855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:53.904855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:53.904855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:53.904855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:53.904855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:53.904855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:53.904855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:53.905951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:53.905951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:53.905951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.905951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:53.911746 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:53.912035 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:53.912188 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:53.912351 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:53.912961 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:53.913080 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:53.913203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.914077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:53.914077 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:53.914077 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:53.914077 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:53.914077 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:53.914077 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:53.914077 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.914077 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:53.915909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.915909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:53.915909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:53.915909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:53.915909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.915909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:23:53.915909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:53.915909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:53.915909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:53.915909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:53.915909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:53.915909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:53.916439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:53.916439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:53.916439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:53.916439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:53.916439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:53.916439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:53.916439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:53.916439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:53.916439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:53.916439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:53.916439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:53.916439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:53.916439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:53.917027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.917027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:53.917808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.917808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:53.917808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:53.917808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:53.917808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:53.917808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:53.917808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.917808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:53.917808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:53.917808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.917808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:53.917808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:53.917808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.917808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:53.917808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:53.917808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:53.918623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:23:53.918623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:53.918623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:53.918623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:53.918623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:53.918623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:53.918623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:53.918623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:53.918623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:53.918623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:53.918623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:53.918623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:53.918623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:53.918623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:53.918623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:53.918623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:53.918623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:53.918623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:53.918623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:53.919521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:53.919521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:53.919521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:58.590447 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:58.591085 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:58.591085 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:58.591943 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:58.627156 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:58.628249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.628249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:58.628249 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:58.628249 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:58.628249 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:58.628249 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:58.628249 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:58.628249 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.628249 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:58.629055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.629055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:58.629055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:58.629055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:58.629055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.629055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:23:58.629055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:58.629055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:58.629055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:58.629055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:58.629055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:58.629055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:58.629568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:58.629568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:58.629568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:58.629568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:58.629568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:58.629568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:58.629568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:58.629568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:58.629568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:58.629568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:58.629568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:58.629568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:58.629568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:58.630035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.630035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:58.630035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.630035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:58.630035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:58.630035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:58.630035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:58.630035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:58.630035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.630035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:58.630035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:58.630035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.630035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:58.630035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:58.630035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.630474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:58.630474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:58.630474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.630474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:23:58.630474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:58.630474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:58.630474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:58.630474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:58.630474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:58.630474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:58.630474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:58.630474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:58.630474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:58.630474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:58.630474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:58.630474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:58.630474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:58.630474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:58.630474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:58.631156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:58.631156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:58.631156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:58.631156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:58.631156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.631156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:58.820475 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:58.820732 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:58.820879 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:58.821892 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:58.822251 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:58.822494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.825709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:58.825709 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:58.825709 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:58.825709 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:58.825709 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:58.825709 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:58.825709 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.825709 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:58.825709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.825709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:58.825709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:58.825709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:58.825709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.825709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:23:58.825709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:58.826427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:58.826427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:58.826427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:58.826427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:58.826427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:58.826427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:58.826427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:58.826427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:58.826427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:58.826427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:58.826427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:58.826427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:58.826427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:58.826427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:58.826878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:58.826878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:58.826878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:58.826878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:58.826878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.826878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:58.826878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.826878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:58.826878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:58.826878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:58.826878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:58.826878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:58.826878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.826878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:58.827371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:58.827371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.827371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:58.827371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:58.827371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.827371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:58.827371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:58.827371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.827371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:23:58.827371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:58.827371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:58.827371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:58.827371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:58.827371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:58.827371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:58.827371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:58.827371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:58.827371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:58.827371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:58.827371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:58.828216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:58.828216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:58.828216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:58.828216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:58.828216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:58.828216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:58.828216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:58.828216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:58.828216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.828216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:58.882980 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:58.883290 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:58.883417 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:58.883551 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:23:58.884323 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:58.884541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.885219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:58.885219 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:58.885219 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:58.885219 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:58.885219 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:58.885219 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:58.885219 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.885219 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:58.886424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.886424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:23:58.886424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:58.886424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:58.886424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:58.886424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:58.886424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:58.886424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:58.886424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:58.886424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:58.886424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:58.886424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:23:58.886424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:58.886424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:23:58.886424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:23:58.888027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:58.888027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:23:58.888027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:58.888027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:23:58.888027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:23:58.888027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:58.888027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:23:58.888027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.888027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:23:58.888501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.888501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:58.888501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:58.888501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:58.888501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.888501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:23:58.888501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:23:58.888501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:23:58.888501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:58.888501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:58.888501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:58.888501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:58.890204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:58.890204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:58.890204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:58.890204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:58.890204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:58.890204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:58.890204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:58.890204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:58.890204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:58.890204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:58.890204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:58.890204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:58.890204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:58.890204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:58.891059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:58.891059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:58.891059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.891059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:58.891059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.891059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:58.891059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:58.891059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:58.891059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:58.891059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:58.891059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.891059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:58.891059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:58.891059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.891668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:58.891668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:58.891668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.891668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:58.891668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:58.891668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.891668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:23:58.891668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:23:58.891668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:23:58.891668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:58.891668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:58.891668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:58.891668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:58.891668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:58.891668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:58.891668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:58.891668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:58.891668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:58.891668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:58.892611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:58.892611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:58.892611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:58.892611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:58.892611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:58.892611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:58.892611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:58.892611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:58.892611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:58.892611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:58.892611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.892611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:58.899186 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:58.899532 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:58.899719 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:58.899886 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:58.900506 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:58.900774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.901416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:58.901416 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:58.901416 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:58.901416 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:58.901416 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:58.901416 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:58.901416 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.901416 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:58.902327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.902327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:23:58.902327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:58.902327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:58.902327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:58.902327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:58.902327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:58.902327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:58.902327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:58.902327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:23:58.902327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:58.902327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:23:58.902327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:58.902327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:23:58.902327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:23:58.903233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:58.903233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:23:58.903233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:58.903233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:23:58.903233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:23:58.903233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:58.903233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:23:58.903233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.903233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:23:58.903763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.903763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:58.903763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:58.903763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:58.903763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.903763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:23:58.903763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:23:58.903763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:58.903763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:58.903763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:58.903763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:58.904306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:58.904306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:58.904306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:58.904306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:58.904306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:58.904306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:58.904306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:58.904306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:58.904306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:58.904306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:58.904306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:58.904306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:58.904306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:58.904306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:58.904939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:58.904939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:58.904939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.904939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:58.904939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.904939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:58.904939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:58.904939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:58.904939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:58.904939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:58.904939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.904939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:58.904939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:58.904939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.905663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:58.905663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:58.905663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.905663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:58.905663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:58.905663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.905663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:23:58.905663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:23:58.905663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:23:58.905663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:23:58.905663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:58.905663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:58.905663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:58.905663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:58.905663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:58.905663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:58.905663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:58.905663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:58.905663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:58.906521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:58.906521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:58.906521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:58.906521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:58.906521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:58.906521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:58.906521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:58.906521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:58.906521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:58.906521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.906521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:23:58.913062 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:23:58.913558 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:23:58.913869 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:23:58.914240 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:23:58.914646 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:58.914951 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:23:58.915103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.915831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:23:58.915831 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:58.915831 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:23:58.915831 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:23:58.915831 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:58.915831 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:23:58.915831 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.915831 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:23:58.917436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.917436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:23:58.917436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:58.917436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:23:58.917436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.917436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:23:58.917436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:58.917436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:58.917436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:58.917436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:58.917436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:58.917436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:58.918338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:23:58.918338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:58.918338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:23:58.918338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:58.918338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:23:58.918338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:23:58.918338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:58.918338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:23:58.918338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:58.918338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:23:58.918338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:23:58.918338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:58.918338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:23:58.918945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.918945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:23:58.918945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.918945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:23:58.918945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:23:58.918945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:23:58.918945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:23:58.918945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:23:58.918945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.918945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:23:58.918945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:23:58.918945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.918945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:23:58.918945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:23:58.918945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.919572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:23:58.919572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:23:58.919572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:23:58.919572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:23:58.919572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:23:58.919572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:58.919572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:58.919572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:23:58.919572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:23:58.919572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:23:58.919572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:23:58.919572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:23:58.919572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:23:58.919572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:23:58.919572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:23:58.919572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:23:58.919572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:23:58.919572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:23:58.919572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:23:58.920722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:23:58.920722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:23:58.920722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:23:58.920722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:23:58.920722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:23:58.920722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:03.590983 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:03.591501 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:03.591710 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:03.593062 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:03.594745 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=238 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:24:03.629135 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:03.629472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.630239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:03.630239 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:03.630239 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:03.630239 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:03.630239 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:03.630239 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:03.630239 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.630239 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:03.630899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.630899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:03.630899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:03.630899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:03.630899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.630899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:24:03.630899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:03.630899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:03.630899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:03.630899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:03.630899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:03.630899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:03.631482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:03.631482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:03.631482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:03.631482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:03.631482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:03.631482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:03.631482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:03.631482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:03.631482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:03.631482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:03.631482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:03.631482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:03.631482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:03.632127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.632127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:03.632127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.632127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:03.632127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:03.632127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:03.632127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:03.632127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:03.632127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.632127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:03.632127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:03.632127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.632127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:03.632127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:03.632127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.632939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:03.632939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:03.632939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.632939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:24:03.632939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:03.632939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:03.632939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:03.632939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:03.632939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:03.632939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:03.632939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:03.632939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:03.632939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:03.632939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:03.632939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:03.632939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:03.632939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:03.632939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:03.632939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:03.634164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:03.634164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:03.634164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:03.634164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:03.634164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.634164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:03.822418 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:03.822883 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:03.822978 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:03.823886 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:03.824141 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:03.824311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.825000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:03.825000 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:03.825000 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:03.825000 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:03.825000 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:03.825000 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:03.825000 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.825000 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:03.825928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.825928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:03.825928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:03.825928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:03.825928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.825928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:24:03.825928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:03.825928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:03.825928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:03.825928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:03.825928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:03.825928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:03.826348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:03.826348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:03.826348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:03.826348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:03.826348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:03.826348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:03.826348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:03.826348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:03.826348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:03.826348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:03.826348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:03.826348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:03.826348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:03.826699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.826699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:03.826699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.826699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:03.826699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:03.826699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:03.826699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:03.826699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:03.826699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.826699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:03.826699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:03.826699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.826699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:03.826699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:03.826699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.827112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:03.827112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:03.827112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.827112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:24:03.827112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:03.827112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:03.827112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:03.827112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:03.827112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:03.827112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:03.827112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:03.827112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:03.827112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:03.827112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:03.827112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:03.827112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:03.827112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:03.827112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:03.827112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:03.827596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:03.827596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:03.827596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:03.827596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:03.827596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.827596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:03.883487 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:03.883951 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:03.884065 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:03.884200 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:24:03.885027 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:03.885349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.886125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:03.886125 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:03.886125 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:03.886125 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:03.886125 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:03.886125 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:03.886125 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.886125 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:03.886881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.886881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:24:03.886881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:03.886881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:03.886881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:03.886881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:03.886881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:03.886881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:03.886881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:03.886881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:03.886881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:03.886881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:24:03.886881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:03.886881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:24:03.886881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:24:03.887782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:03.887782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:24:03.887782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:03.887782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:24:03.887782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:24:03.887782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:03.887782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:24:03.887782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.887782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:24:03.887782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.887782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:03.887782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:03.887782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:03.887782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.888826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:24:03.888826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:24:03.888826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:24:03.888826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:03.888826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:03.888826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:03.888826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:03.888826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:03.888826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:03.888826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:03.888826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:03.888826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:03.888826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:03.890412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:03.890412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:03.890412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:03.890412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:03.890412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:03.890412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:03.890412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:03.890412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:03.890412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:03.890412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:03.890412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.890412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:03.891084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.891084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:03.891084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:03.891084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:03.891084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:03.891084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:03.891084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.891084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:03.891084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:03.891084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.891084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:03.891084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:03.891084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.891084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:03.891084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:03.891084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.891890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:24:03.891890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:24:03.891890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:24:03.891890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:03.891890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:03.891890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:03.891890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:03.891890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:03.891890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:03.891890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:03.891890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:03.891890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:03.891890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:03.891890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:03.891890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:03.891890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:03.891890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:03.891890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:03.892777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:03.892777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:03.892777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:03.892777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:03.892777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:03.892777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.892777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:03.899791 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:03.900143 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:03.900240 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:03.900356 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:03.900773 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:03.901124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.901693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:03.901693 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:03.901693 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:03.901693 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:03.901693 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:03.901693 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:03.901693 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.901693 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:03.902418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.902418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:24:03.902418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:03.902418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:03.902418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:03.902418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:03.902418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:03.902418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:03.902418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:03.902418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:03.902418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:03.902418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:24:03.902418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:03.902418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:24:03.902418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:24:03.903200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:03.903200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:24:03.903200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:03.903200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:24:03.903200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:24:03.903200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:03.903200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:24:03.903200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.903200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:24:03.903627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.903627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:03.903627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:03.903627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:03.903627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.903627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:24:03.903627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:24:03.903627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:03.903627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:03.903627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:03.903627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:03.904202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:03.904202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:03.904202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:03.904202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:03.904202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:03.904202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:03.904202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:03.904202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:03.904202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:03.904202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:03.904202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:03.904202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:03.904202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:03.904202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:03.904945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:03.904945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:03.904945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.904945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:03.904945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.904945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:03.904945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:03.904945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:03.904945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:03.904945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:03.904945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.904945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:03.904945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:03.904945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.905637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:03.905637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:03.905637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.905637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:03.905637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:03.905637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.905637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:24:03.905637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:24:03.905637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:03.905637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:03.905637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:03.905637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:03.905637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:03.905637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:03.905637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:03.905637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:03.905637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:03.905637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:03.905637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:03.906724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:03.906724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:03.906724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:03.906724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:03.906724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:03.906724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:03.906724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:03.906724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:03.906724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:03.906724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.906724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:03.913554 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:03.914009 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:03.914198 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:03.914454 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:03.914949 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:03.915297 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:03.915353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.915916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:03.915916 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:03.915916 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:03.915916 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:03.915916 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:03.915916 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:03.915916 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.915916 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:03.917147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.917147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:03.917147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:03.917147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:03.917147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.917147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:24:03.917147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:03.917147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:03.917147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:03.917147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:03.917147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:03.917147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:03.917757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:03.917757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:03.917757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:03.917757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:03.917757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:03.917757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:03.917757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:03.917757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:03.917757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:03.917757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:03.917757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:03.917757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:03.917757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:03.918181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.918181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:03.918480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.918480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:03.918480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:03.918480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:03.918480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:03.918480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:03.918480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.918480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:03.918480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:03.918480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.918480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:03.918480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:03.918480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.918480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:03.918480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:03.918480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:03.918962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:24:03.918962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:03.918962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:03.918962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:03.918962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:03.918962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:03.918962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:03.918962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:03.918962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:03.918962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:03.918962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:03.918962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:03.918962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:03.918962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:03.918962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:03.918962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:03.918962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:03.918962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:03.918962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:03.919456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:03.919456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:03.919456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:08.592793 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:08.592793 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:08.592793 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:08.593281 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:08.624860 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:08.625220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.626185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:08.626185 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:08.626185 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:08.626185 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:08.626185 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:08.626185 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:08.626185 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.626185 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:08.627000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.627000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:08.627000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:08.627000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:08.627000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.627000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:24:08.627000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:08.627000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:08.627000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:08.627000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:08.627000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:08.627000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:08.627380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:08.627380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:08.627380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:08.627380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:08.627380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:08.627380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:08.627380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:08.627380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:08.627380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:08.627380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:08.627380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:08.627380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:08.627380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:08.627771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.627771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:08.627771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.627771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:08.627771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:08.627771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:08.627771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:08.627771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:08.627771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.627771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:08.627771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:08.627771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.627771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:08.627771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:08.627771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.628216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:08.628216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:08.628216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.628216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:24:08.628216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:08.628216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:08.628216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:08.628216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:08.628216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:08.628216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:08.628216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:08.628216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:08.628216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:08.628216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:08.628216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:08.628216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:08.628216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:08.628216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:08.628216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:08.628743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:08.628743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:08.628743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:08.628743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:08.628743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.628743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:08.823043 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:08.823555 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:08.823727 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:08.824638 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:08.824985 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:08.825206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.826108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:08.826108 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:08.826108 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:08.826108 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:08.826108 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:08.826108 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:08.826108 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.826108 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:08.828088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.828088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:08.828088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:08.828088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:08.828088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.828088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:24:08.828088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:08.828088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:08.828088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:08.828088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:08.828088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:08.828088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:08.828871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:08.828871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:08.828871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:08.828871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:08.828871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:08.828871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:08.828871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:08.828871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:08.828871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:08.828871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:08.828871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:08.828871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:08.828871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:08.829957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.829957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:08.831920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.831920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:08.831920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:08.831920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:08.831920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:08.831920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:08.831920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.831920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:08.831920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:08.831920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.831920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:08.831920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:08.831920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.831920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:08.831920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:08.831920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.832672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:24:08.832672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:08.832672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:08.832672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:08.832672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:08.832672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:08.832672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:08.832672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:08.832672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:08.832672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:08.832672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:08.832672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:08.832672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:08.832672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:08.832672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:08.832672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:08.832672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:08.832672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:08.832672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:08.833669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:08.833669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.833669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:08.883975 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:08.884393 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:08.884523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:08.884652 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:24:08.885409 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:08.885409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.885691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:08.885691 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:08.885691 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:08.885691 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:08.885691 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:08.885691 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:08.885691 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.885691 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:08.886612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.886612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:24:08.886612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:08.886612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:08.886612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:08.886612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:08.886612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:08.886612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:08.886612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:08.886612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:08.886612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:08.886612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:24:08.886612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:08.886612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:24:08.886612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:24:08.887025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:08.887025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:24:08.887025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:08.887025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:24:08.887025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:24:08.887025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:08.887025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:24:08.887025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.887025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:24:08.887839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.887839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:08.887839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:08.887839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:08.887839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.887839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:24:08.887839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:24:08.887839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:24:08.887839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:08.887839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:08.887839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:08.887839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:08.888227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:08.888227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:08.888227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:08.888227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:08.888227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:08.888227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:08.888227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:08.888227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:08.888227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:08.888227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:08.888227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:08.888227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:08.888227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:08.888227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:08.888473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:08.888473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:08.888473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.888473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:08.890702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.890702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:08.890702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:08.890702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:08.890702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:08.890702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:08.890702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.890702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:08.890702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:08.890702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.890702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:08.890702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:08.890702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.890702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:08.890702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:08.890702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.891098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:24:08.891098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:24:08.891098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:24:08.891098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:08.891098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:08.891098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:08.891098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:08.891098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:08.891098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:08.891098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:08.891098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:08.891098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:08.891098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:08.891098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:08.891098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:08.891098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:08.891098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:08.891098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:08.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:08.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:08.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:08.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:08.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:08.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.891799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:08.900278 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:08.900567 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:08.900703 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:08.900858 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:08.901226 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:08.901444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.902004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:08.902004 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:08.902004 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:08.902004 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:08.902004 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:08.902004 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:08.902004 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.902004 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:08.902894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.902894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:24:08.902894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:08.902894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:08.902894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:08.902894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:08.902894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:08.902894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:08.902894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:08.902894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:08.902894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:08.902894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:24:08.902894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:08.902894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:24:08.902894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:24:08.903293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:08.903293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:24:08.903293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:08.903293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:24:08.903293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:24:08.903293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:08.903293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:24:08.903293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.903293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:24:08.903807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.903807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:08.903807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:08.903807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:08.903807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.903807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:24:08.903807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:24:08.903807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:08.903807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:08.903807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:08.903807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:08.904145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:08.904145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:08.904145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:08.904145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:08.904145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:08.904145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:08.904145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:08.904145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:08.904145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:08.904145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:08.904145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:08.904145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:08.904145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:08.904145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:08.904514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:08.904514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:08.904514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.904514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:08.905257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.905257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:08.905257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:08.905257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:08.905257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:08.905257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:08.905257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.905257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:08.905257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:08.905257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.905257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:08.905257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:08.905257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.905257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:08.905257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:08.905257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.905257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:24:08.905738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:24:08.905738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:08.905738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:08.905738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:08.905738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:08.905738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:08.905738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:08.905738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:08.905738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:08.905738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:08.905738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:08.905738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:08.905738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:08.905738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:08.905738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:08.905738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:08.905738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:08.905738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:08.905738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:08.906214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:08.906214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:08.906214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.906214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:08.914213 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:08.914345 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:08.914571 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:08.914759 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:08.915133 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:08.915411 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:08.915583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.916943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:08.916943 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:08.916943 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:08.916943 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:08.916943 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:08.916943 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:08.916943 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.916943 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:08.917963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.917963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:08.917963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:08.917963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:08.917963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.917963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:24:08.917963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:08.917963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:08.917963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:08.917963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:08.917963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:08.917963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:08.918296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:08.918296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:08.918296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:08.918296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:08.918296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:08.918296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:08.918296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:08.918296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:08.918296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:08.918296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:08.918296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:08.918296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:08.918296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:08.918699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.918699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:08.919063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.919063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:08.919063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:08.919063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:08.919063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:08.919063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:08.919063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.919063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:08.919063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:08.919063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.919063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:08.919063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:08.919063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.919063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:08.919063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:08.919063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:08.919453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:24:08.919453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:08.919453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:08.919453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:08.919453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:08.919453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:08.919453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:08.919453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:08.919453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:08.919453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:08.919453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:08.919453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:08.919453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:08.919453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:08.919453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:08.919453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:08.919453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:08.919453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:08.919453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:08.919953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:08.919953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:08.919953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:13.592225 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:13.592980 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:13.592980 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:13.593488 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:13.624163 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:13.624983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.625097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:13.625097 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:13.625097 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:13.625097 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:13.625097 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:13.625097 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:13.625097 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.625097 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:13.626024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.626024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:13.626024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:13.626024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:13.626024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.626024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:24:13.626024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:13.626024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:13.626024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:13.626024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:13.626024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:13.626024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:13.626439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:13.626439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:13.626439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:13.626439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:13.626439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:13.626439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:13.626439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:13.626439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:13.626439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:13.626439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:13.626439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:13.626439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:13.626439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:13.626820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.626820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:13.627002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.627002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:13.627002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:13.627002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:13.627002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:13.627002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:13.627002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.627002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:13.627002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:13.627002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.627002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:13.627002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:13.627002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.627002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:13.627002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:13.627002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.627627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:24:13.627627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:13.627627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:13.627627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:13.627627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:13.627627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:13.627627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:13.627627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:13.627627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:13.627627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:13.627627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:13.627627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:13.627627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:13.627627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:13.627627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:13.627627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:13.627627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:13.627627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:13.627627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:13.628260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:13.628260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.628260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:13.823608 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:13.824322 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:13.824499 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:13.825401 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:13.825585 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:13.825722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.826436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:13.826436 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:13.826436 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:13.826436 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:13.826436 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:13.826436 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:13.826436 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.826436 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:13.828147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.828147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:13.828147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:13.828147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:13.828147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.828147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:24:13.828147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:13.828147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:13.828147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:13.828147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:13.828147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:13.828147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:13.828774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:13.828774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:13.828774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:13.828774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:13.828774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:13.828774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:13.828774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:13.828774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:13.828774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:13.828774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:13.828774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:13.828774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:13.828774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:13.829297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.829297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:13.829696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.829696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:13.829696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:13.829696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:13.829696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:13.829696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:13.829696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.829696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:13.829696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:13.829696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.829696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:13.829696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:13.829696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.829696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:13.829696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:13.829696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.830176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:24:13.830176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:13.830176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:13.830176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:13.830176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:13.830176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:13.830176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:13.830176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:13.830176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:13.830176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:13.830176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:13.830176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:13.830176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:13.830176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:13.830176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:13.830176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:13.830176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:13.830176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:13.830176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:13.830723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:13.830723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.830723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:13.884506 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:13.884962 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:13.885494 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:13.885494 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:24:13.885774 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:13.886196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.886559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:13.886559 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:13.886559 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:13.886559 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:13.886559 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:13.886559 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:13.886559 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.886559 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:13.887830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.887830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:24:13.887830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:13.887830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:13.887830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:13.887830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:13.887830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:13.887830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:13.887830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:13.887830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:13.887830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:13.887830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:24:13.887830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:13.887830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:24:13.887830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:24:13.888179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:13.888179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:24:13.888179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:13.888179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:24:13.888179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:24:13.888179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:13.888179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:24:13.888179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.888179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:24:13.889334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.889334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:13.889334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:13.889334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:13.889334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.889334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:24:13.889334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:24:13.889334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:24:13.889334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:13.889334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:13.889334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:13.889334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:13.889601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:13.889601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:13.889601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:13.889601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:13.889601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:13.889601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:13.889601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:13.889601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:13.889601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:13.889601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:13.889601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:13.889601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:13.889601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:13.889601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:13.889869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:13.889869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:13.889869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.889869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:13.891221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.891221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:13.891221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:13.891221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:13.891221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:13.891221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:13.891221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.891221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:13.891221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:13.891221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.891221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:13.891221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:13.891221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.891221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:13.891221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:13.891221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.891542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:24:13.891542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:24:13.891542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:24:13.891542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:13.891542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:13.891542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:13.891542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:13.891542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:13.891542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:13.891542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:13.891542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:13.891542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:13.891542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:13.891542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:13.891542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:13.891542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:13.891542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:13.891542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:13.891907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:13.891907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:13.891907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:13.891907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:13.891907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:13.891907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.891907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:13.900803 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:13.901513 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:13.901638 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:13.901819 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:13.902266 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:13.902546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.903178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:13.903178 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:13.903178 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:13.903178 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:13.903178 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:13.903178 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:13.903178 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.903178 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:13.905852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.905852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:24:13.905852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:13.905852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:13.905852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:13.905852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:13.905852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:13.905852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:13.905852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:13.905852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:13.905852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:13.905852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:24:13.905852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:13.905852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:24:13.905852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:24:13.906361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:13.906361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:24:13.906361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:13.906361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:24:13.906361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:24:13.906361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:13.906361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:24:13.906361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.906361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:24:13.906361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.906361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:13.906361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:13.906361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:13.906361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.906820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:24:13.906820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:24:13.906820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:13.906820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:13.906820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:13.906820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:13.906820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:13.906820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:13.906820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:13.906820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:13.906820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:13.906820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:13.906820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:13.907164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:13.907164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:13.907164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:13.907164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:13.907164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:13.907164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:13.907164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:13.907164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:13.907164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:13.907164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.907164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:13.908113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.908113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:13.908113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:13.908113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:13.908113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:13.908113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:13.908113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.908113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:13.908113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:13.908113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.908113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:13.908113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:13.908113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.908113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:13.908113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:13.908113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.908113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:24:13.908594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:24:13.908594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:13.908594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:13.908594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:13.908594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:13.908594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:13.908594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:13.908594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:13.908594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:13.908594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:13.908594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:13.908594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:13.908594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:13.908594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:13.908594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:13.908594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:13.908594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:13.908594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:13.908594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:13.909140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:13.909140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:13.909140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.909140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:13.914515 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:13.914912 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:13.915038 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:13.915291 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:13.915743 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:13.916099 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:13.916319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.917226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:13.917226 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:13.917226 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:13.917226 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:13.917226 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:13.917226 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:13.917226 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.917226 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:13.918382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.918382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:13.918382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:13.918382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:13.918382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.918382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:24:13.918382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:13.918382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:13.918382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:13.918382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:13.918382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:13.918382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:13.918763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:13.918763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:13.918763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:13.918763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:13.918763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:13.918763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:13.918763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:13.918763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:13.918763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:13.918763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:13.918763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:13.918763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:13.918763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:13.919099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.919099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:13.919559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.919559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:13.919559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:13.919559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:13.919559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:13.919559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:13.919559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.919559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:13.919559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:13.919559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.919559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:13.919559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:13.919559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.919559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:13.919559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:13.919559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:13.919993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:24:13.919993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:13.919993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:13.919993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:13.919993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:13.919993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:13.919993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:13.919993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:13.919993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:13.919993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:13.919993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:13.919993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:13.919993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:13.919993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:13.919993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:13.919993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:13.919993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:13.919993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:13.919993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:13.920480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:13.920480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:13.920480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:18.592824 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:18.593473 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:18.593473 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:18.594378 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:18.629063 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:18.629362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.630401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:18.630401 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:18.630401 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:18.630401 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:18.630401 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:18.630401 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:18.630401 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.630401 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:18.631151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.631151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:18.631151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:18.631151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:18.631151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.631151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:24:18.631151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:18.631151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:18.631151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:18.631151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:18.631151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:18.631151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:18.632441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:18.632441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:18.632441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:18.632441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:18.632441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:18.632441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:18.632441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:18.632441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:18.632441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:18.632441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:18.632441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:18.632441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:18.632441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:18.633117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.633117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:18.633117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.633117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:18.633117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:18.633117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:18.633117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:18.633117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:18.633117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.633117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:18.633117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:18.633117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.633117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:18.633117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:18.633117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.635090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:18.635090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:18.635090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.635090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:24:18.635090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:18.635090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:18.635090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:18.635090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:18.635090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:18.635090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:18.635090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:18.635090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:18.635090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:18.635090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:18.635090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:18.635090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:18.635090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:18.635090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:18.635090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:18.636319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:18.636319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:18.636319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:18.636319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:18.636319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.636319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:18.827093 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:18.827519 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:18.827642 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:18.828375 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:18.828591 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:18.828749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.829281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:18.829281 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:18.829281 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:18.829281 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:18.829281 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:18.829281 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:18.829281 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.829281 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:18.830091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.830091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:18.830091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:18.830091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:18.830091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.830091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:24:18.830091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:18.830091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:18.830091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:18.830091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:18.830091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:18.830091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:18.830563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:18.830563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:18.830563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:18.830563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:18.830563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:18.830563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:18.830563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:18.830563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:18.830563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:18.830563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:18.830563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:18.830563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:18.830563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:18.831192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.831192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:18.831192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.831192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:18.831192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:18.831192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:18.831192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:18.831192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:18.831192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.831192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:18.831192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:18.831192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.831192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:18.831192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:18.831192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.831624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:18.831624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:18.831624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.831624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:24:18.831624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:18.831624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:18.831624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:18.831624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:18.831624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:18.831624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:18.831624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:18.831624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:18.831624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:18.831624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:18.831624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:18.831624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:18.831624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:18.831624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:18.831624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:18.832282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:18.832282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:18.832282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:18.832282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:18.832282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.832282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:18.885190 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:18.885559 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:18.885752 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:18.885815 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:24:18.887278 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:18.887278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.887278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:18.887278 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:18.887278 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:18.887278 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:18.887278 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:18.887278 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:18.887278 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.887278 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:18.888012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.888012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:24:18.888012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:18.888012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:18.888012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:18.888012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:18.888012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:18.888012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:18.888012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:18.888012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:18.888012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:18.888012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:24:18.888012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:18.888012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:24:18.888012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:24:18.890337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:18.890337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:24:18.890337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:18.890337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:24:18.890337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:24:18.890337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:18.890337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:24:18.890337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.890337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:24:18.890337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.890337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:18.890337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:18.890337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:18.890337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.891462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:24:18.891462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:24:18.891462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:24:18.891462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:18.891462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:18.891462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:18.891462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:18.891462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:18.891462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:18.891462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:18.891462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:18.891462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:18.891462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:18.891885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:18.891885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:18.891885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:18.891885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:18.891885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:18.891885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:18.891885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:18.891885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:18.891885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:18.891885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:18.891885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.891885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:18.892098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.892098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:18.892098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:18.892098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:18.892098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:18.892098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:18.892098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.892098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:18.892098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:18.892098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.892098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:18.892098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:18.892098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.892098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:18.892098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:18.892098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.892388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:24:18.892388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:24:18.892388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:24:18.892388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:18.892388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:18.892388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:18.892388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:18.892388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:18.892388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:18.892388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:18.892388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:18.892388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:18.892388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:18.892388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:18.892388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:18.892388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:18.892388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:18.892388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:18.894505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:18.894505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:18.894505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:18.894505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:18.894505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:18.894505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.894505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:18.901613 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:18.901922 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:18.902026 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:18.902157 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:18.903050 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:18.903310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.903956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:18.903956 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:18.903956 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:18.903956 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:18.903956 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:18.903956 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:18.903956 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.903956 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:18.904814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.904814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:24:18.904814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:18.904814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:18.904814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:18.904814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:18.904814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:18.904814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:18.904814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:18.904814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:18.904814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:18.904814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:24:18.904814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:18.904814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:24:18.904814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:24:18.905734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:18.905734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:24:18.905734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:18.905734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:24:18.905734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:24:18.905734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:18.905734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:24:18.905734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.905734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:24:18.905734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.905734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:18.905734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:18.905734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:18.905734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.906150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:24:18.906150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:24:18.906150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:18.906150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:18.906150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:18.906150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:18.906150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:18.906150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:18.906150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:18.906150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:18.906150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:18.906150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:18.906150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:18.906478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:18.906478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:18.906478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:18.906478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:18.906478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:18.906478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:18.906478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:18.906478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:18.906478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:18.906478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.906478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:18.907066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.907066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:18.907066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:18.907066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:18.907066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:18.907066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:18.907066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.907066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:18.907066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:18.907066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.907066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:18.907066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:18.907066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.907066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:18.907066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:18.907066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.907066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:24:18.908487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:24:18.908487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:18.908487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:18.908487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:18.908487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:18.908487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:18.908487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:18.908487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:18.908487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:18.908487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:18.908487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:18.908487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:18.908487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:18.908487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:18.908487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:18.908487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:18.908487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:18.908487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:18.908487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:18.909520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:18.909520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:18.909520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.909520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:18.914951 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:18.915579 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:18.915754 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:18.915945 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:18.916379 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:18.916617 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:18.916790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.917373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:18.917373 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:18.917373 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:18.917373 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:18.917373 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:18.917373 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:18.917373 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.917373 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:18.918510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.918510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:18.918510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:18.918510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:18.918510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.918510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:24:18.918510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:18.918510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:18.918510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:18.918510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:18.918510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:18.918510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:18.918873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:18.918873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:18.918873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:18.918873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:18.918873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:18.918873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:18.918873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:18.918873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:18.918873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:18.918873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:18.918873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:18.918873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:18.918873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:18.919222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.919222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:18.920002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.920002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:18.920002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:18.920002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:18.920002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:18.920002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:18.920002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.920002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:18.920002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:18.920002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.920002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:18.920002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:18.920002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.920002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:18.920002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:18.920002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:18.920440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:24:18.920440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:18.920440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:18.920440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:18.920440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:18.920440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:18.920440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:18.920440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:18.920440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:18.920440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:18.920440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:18.920440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:18.920440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:18.920440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:18.920440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:18.920440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:18.920440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:18.920440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:18.920440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:18.920964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:18.920964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:18.920964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:23.593389 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:23.594310 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:23.594310 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:23.594572 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:23.620180 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:23.620377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.620929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:23.620929 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:23.620929 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:23.620929 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:23.620929 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:23.620929 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:23.620929 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.620929 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:23.621646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.027 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.621646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:23.621646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:23.621646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:23.621646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.621646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:24:23.621646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:23.621646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:23.621646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:23.621646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:23.621646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:23.621646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:23.622148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:23.622148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:23.622148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:23.622148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:23.622148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:23.622148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:23.622148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:23.622148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:23.622148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:23.622148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:23.622148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:23.622148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:23.622148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:23.622618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.622618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:23.622618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.622618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:23.622618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:23.622618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:23.622618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:23.622618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:23.622618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.622618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:23.622618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:23.622618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.622618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:23.622618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:23.622618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.623139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:23.623139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:23.623139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.623139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:24:23.623139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:23.623139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:23.623139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:23.623139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:23.623139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:23.623139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:23.623139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:23.623139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:23.623139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:23.623139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:23.623139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:23.623139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:23.623139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:23.623139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:23.623139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:23.623738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:23.623738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:23.623738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:23.623738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:23.623738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.623738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:23.828343 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:23.828814 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:23.829236 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:23.830060 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:23.830540 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:23.830540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.831117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:23.831117 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:23.831117 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:23.831117 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:23.831117 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:23.831117 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:23.831117 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.831117 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:23.832979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.832979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:23.832979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:23.832979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:23.832979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.832979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:24:23.832979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:23.832979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:23.832979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:23.832979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:23.832979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:23.832979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:23.834243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:23.834243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:23.834243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:23.834243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:23.834243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:23.834243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:23.834243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:23.834243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:23.834243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:23.834243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:23.834243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:23.834243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:23.834243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:23.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:23.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:23.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:23.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:23.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:23.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:23.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:23.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:23.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:23.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:23.835009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.835627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:23.835627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:23.835627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.835627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:24:23.835627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:23.835627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:23.835627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:23.835627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:23.835627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:23.835627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:23.835627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:23.835627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:23.835627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:23.835627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:23.835627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:23.835627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:23.835627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:23.835627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:23.835627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:23.836038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:23.836038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:23.836038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:23.836038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:23.836038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.836038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:23.885830 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:23.886139 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:23.886206 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:23.886319 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:24:23.887767 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:23.888369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.889399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:23.889399 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:23.889399 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:23.889399 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:23.889399 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:23.889399 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:23.889399 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.889399 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:23.889992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.889992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:24:23.889992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:23.889992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:23.889992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:23.889992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:23.889992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:23.889992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:23.889992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:23.889992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:23.889992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:23.889992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:24:23.889992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:23.889992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:24:23.889992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:24:23.890390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:23.890390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:24:23.890390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:23.890390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:24:23.890390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:24:23.890390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:23.890390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:24:23.890390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.890390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:24:23.890591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.890591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:23.890591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:23.890591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:23.890591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.890591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:24:23.890591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:24:23.890591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:24:23.890591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:23.890591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:23.890591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:23.890591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:23.890915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:23.890915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:23.890915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:23.890915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:23.890915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:23.890915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:23.890915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:23.890915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:23.890915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:23.890915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:23.890915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:23.890915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:23.890915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:23.890915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:23.891201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:23.891201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:23.891201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.891201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:23.891338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.891338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:23.891338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:23.891338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:23.891338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:23.891338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:23.891338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.891338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:23.891338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:23.891338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.891338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:23.891338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:23.891338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.891338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:23.891338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:23.891338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.891755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:24:23.891755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:24:23.891755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:24:23.891755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:23.891755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:23.891755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:23.891755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:23.891755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:23.891755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:23.891755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:23.891755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:23.891755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:23.891755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:23.891755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:23.891755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:23.891755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:23.891755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:23.891755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:23.892141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:23.892141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:23.892141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:23.892141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:23.892141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:23.892141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.892141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:23.902064 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:23.902459 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:23.902645 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:23.902868 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:23.903707 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:23.904218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.905159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:23.905159 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:23.905159 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:23.905159 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:23.905159 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:23.905159 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:23.905159 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.905159 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:23.906350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.906350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:24:23.906350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:23.906350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:23.906350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:23.906350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:23.906350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:23.906350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:23.906350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:23.906350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:23.906350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:23.906350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:24:23.906350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:23.906350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:24:23.906350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:24:23.907243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:23.907243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:24:23.907243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:23.907243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:24:23.907243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:24:23.907243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:23.907243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:24:23.907243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.907243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:24:23.907891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.907891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:23.907891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:23.907891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:23.907891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.907891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:24:23.907891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:24:23.907891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:23.907891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:23.907891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:23.907891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:23.908543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:23.908543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:23.908543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:23.908543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:23.908543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:23.908543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:23.908543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:23.908543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:23.908543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:23.908543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:23.908543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:23.908543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:23.908543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:23.908543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:23.909284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:23.909284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:23.909284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.909284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:23.909545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.909545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:23.909545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:23.909545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:23.909545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:23.909545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:23.909545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.909545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:23.909545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:23.909545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.909545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:23.909545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:23.909545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.909545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:23.909545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:23.909545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.909545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:24:23.910484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:24:23.910484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:23.910484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:23.910484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:23.910484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:23.910484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:23.910484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:23.910484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:23.910484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:23.910484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:23.910484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:23.910484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:23.910484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:23.910484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:23.910484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:23.910484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:23.910484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:23.910484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:23.910484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:23.912641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:23.912641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:23.912641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.912641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:23.915632 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:23.916129 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:23.916129 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:23.916219 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:23.917901 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:23.918665 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:23.919481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.922185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:23.922185 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:23.922185 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:23.922185 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:23.922185 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:23.922185 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:23.922185 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.922185 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:23.923133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.923133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:23.923133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:23.923133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:23.923133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.923133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:24:23.923133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:23.923133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:23.923133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:23.923133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:23.923133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:23.923133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:23.923467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:23.923467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:23.923467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:23.923467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:23.923467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:23.923467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:23.923467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:23.923467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:23.923467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:23.923467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:23.923467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:23.923467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:23.923467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:23.923728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.923728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:23.924470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.924470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:23.924470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:23.924470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:23.924470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:23.924470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:23.924470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.924470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:23.924470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:23.924470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.924470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:23.924470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:23.924470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.924470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:23.924470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:23.924470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:23.925116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:24:23.925116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:23.925116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:23.925116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:23.925116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:23.925116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:23.925116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:23.925116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:23.925116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:23.925116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:23.925116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:23.925116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:23.925116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:23.925116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:23.925116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:23.925116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:23.925116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:23.925116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:23.925116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:23.925469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:23.925469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:23.925469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:28.594024 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:28.594626 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:28.594626 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:28.595491 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:28.626355 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:28.626605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.627314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:28.627314 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:28.627314 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:28.627314 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:28.627314 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:28.627314 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:28.627314 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.627314 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:28.628090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.628090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:28.628090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:28.628090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:28.628090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.628090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:24:28.628090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:28.628090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:28.628090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:28.628090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:28.628090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:28.628090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:28.628531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:28.628531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:28.628531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:28.628531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:28.628531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:28.628531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:28.628531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:28.628531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:28.628531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:28.628531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:28.628531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:28.628531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:28.628531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:28.628953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.628953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:28.628953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.628953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:28.628953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:28.628953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:28.628953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:28.628953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:28.628953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.628953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:28.628953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:28.628953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.628953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:28.628953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:28.628953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.629504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:28.629504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:28.629504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.629504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:24:28.629504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:28.629504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:28.629504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:28.629504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:28.629504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:28.629504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:28.629504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:28.629504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:28.629504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:28.629504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:28.629504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:28.629504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:28.629504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:28.629504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:28.629504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:28.630149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:28.630149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:28.630149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:28.630149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:28.630149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.630149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:28.828912 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:28.829704 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:28.829918 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:28.830924 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:28.831143 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:28.831298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.832029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:28.832029 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:28.832029 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:28.832029 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:28.832029 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:28.832029 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:28.832029 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.832029 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:28.833690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.833690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:28.833690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:28.833690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:28.833690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.833690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:24:28.833690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:28.833690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:28.833690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:28.833690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:28.833690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:28.833690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:28.833965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:28.833965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:28.833965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:28.833965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:28.833965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:28.833965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:28.833965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:28.833965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:28.833965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:28.833965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:28.833965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:28.833965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:28.833965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:28.834242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.834242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:28.836108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.836108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:28.836108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:28.836108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:28.836108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:28.836108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:28.836108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.836108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:28.836108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:28.836108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.836108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:28.836108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:28.836108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.836108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:28.836108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:28.836108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.836495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:24:28.836495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:28.836495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:28.836495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:28.836495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:28.836495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:28.836495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:28.836495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:28.836495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:28.836495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:28.836495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:28.836495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:28.836495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:28.836495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:28.836495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:28.836495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:28.836495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:28.836495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:28.836495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:28.836911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:28.836911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.836911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:28.886450 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:28.886958 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:28.887118 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:28.887208 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:24:28.887800 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:28.888038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.888703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:28.888703 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:28.888703 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:28.888703 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:28.888703 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:28.888703 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:28.888703 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.888703 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:28.889925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.889925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:24:28.889925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:28.889925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:28.889925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:28.889925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:28.889925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:28.889925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:28.889925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:28.889925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:28.889925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:28.889925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:24:28.889925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:28.889925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:24:28.889925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:24:28.890825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:28.890825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:24:28.890825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:28.890825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:24:28.890825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:24:28.890825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:28.890825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:24:28.890825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.890825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:24:28.891770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.891770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:28.891770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:28.891770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:28.891770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.891770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:24:28.891770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:24:28.891770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:24:28.891770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:28.891770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:28.891770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:28.891770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:28.892339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:28.892339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:28.892339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:28.892339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:28.892339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:28.892339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:28.892339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:28.892339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:28.892339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:28.892339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:28.892339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:28.892339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:28.892339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:28.892339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:28.892976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:28.892976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:28.892976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.892976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:28.893271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.893271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:28.893271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:28.893271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:28.893271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:28.893271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:28.893271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.893271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:28.893271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:28.893271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.893271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:28.893271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:28.893271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.893271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:28.893271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:28.893271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.894395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:24:28.894395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:24:28.894395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:24:28.894395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:28.894395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:28.894395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:28.894395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:28.894395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:28.894395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:28.894395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:28.894395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:28.894395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:28.894395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:28.894395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:28.894395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:28.894395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:28.894395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:28.894395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:28.895254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:28.895254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:28.895254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:28.895254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:28.895254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:28.895254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.895254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:28.902732 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:28.903259 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:28.903444 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:28.903639 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:28.904124 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:28.904391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.906786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:28.906786 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:28.906786 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:28.906786 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:28.906786 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:28.906786 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:28.906786 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.906786 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:28.908232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.908232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:24:28.908232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:28.908232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:28.908232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:28.908232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:28.908232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:28.908232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:28.908232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:28.908232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:28.908232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:28.908232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:24:28.908232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:28.908232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:24:28.908232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:24:28.908900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:28.908900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:24:28.908900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:28.908900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:24:28.908900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:24:28.908900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:28.908900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:24:28.908900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.908900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:24:28.910241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.910241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:28.910241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:28.910241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:28.910241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.910241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:24:28.910241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:24:28.910241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:28.910241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:28.910241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:28.910241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:28.910804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:28.910804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:28.910804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:28.910804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:28.910804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:28.910804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:28.910804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:28.910804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:28.910804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:28.910804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:28.910804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:28.910804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:28.910804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:28.910804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:28.911347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:28.911347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:28.911347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.911347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:28.912507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.912507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:28.912507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:28.912507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:28.912507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:28.912507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:28.912507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.912507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:28.912507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:28.912507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.912507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:28.912507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:28.912507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.912507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:28.912507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:28.912507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.912507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:24:28.913382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:24:28.913382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:28.913382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:28.913382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:28.913382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:28.913382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:28.913382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:28.913382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:28.913382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:28.913382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:28.913382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:28.913382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:28.913382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:28.913382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:28.913382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:28.913382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:28.913382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:28.913382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:28.913382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:28.914173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:28.914173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:28.914173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.914173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:28.916699 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:28.917087 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:28.917218 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:28.917446 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:28.918126 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:28.918603 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:28.918961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.919901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:28.919901 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:28.919901 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:28.919901 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:28.919901 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:28.919901 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:28.919901 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.919901 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:28.920799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.920799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:28.920799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:28.920799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:28.920799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.920799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:24:28.920799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:28.920799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:28.920799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:28.920799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:28.920799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:28.920799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:28.921239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:28.921239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:28.921239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:28.921239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:28.921239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:28.921239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:28.921239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:28.921239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:28.921239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:28.921239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:28.921239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:28.921239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:28.921239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:28.921614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.921614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:28.921944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.921944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:28.921944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:28.921944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:28.921944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:28.921944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:28.921944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.921944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:28.921944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:28.921944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.921944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:28.921944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:28.921944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.921944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:28.921944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:28.921944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:28.922407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:24:28.922407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:28.922407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:28.922407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:28.922407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:28.922407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:28.922407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:28.922407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:28.922407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:28.922407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:28.922407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:28.922407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:28.922407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:28.922407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:28.922407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:28.922407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:28.922407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:28.922407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:28.922407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:28.922937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:28.922937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:28.922937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:33.594513 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:33.595114 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:33.595114 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:33.595457 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:33.623582 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:33.623876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.624536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:33.624536 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:33.624536 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:33.624536 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:33.624536 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:33.624536 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:33.624536 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.624536 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:33.625353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.625353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:33.625353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:33.625353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:33.625353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.625353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:24:33.625353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:33.625353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:33.625353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:33.625353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:33.625353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:33.625353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:33.625765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:33.625765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:33.625765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:33.625765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:33.625765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:33.625765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:33.625765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:33.625765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:33.625765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:33.625765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:33.625765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:33.625765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:33.625765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:33.626133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.626133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:33.626133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.626133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:33.626133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:33.626133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:33.626133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:33.626133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:33.626133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.626133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:33.626133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:33.626133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.626133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:33.626133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:33.626133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.626526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:33.626526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:33.626526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.626526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:24:33.626526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:33.626526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:33.626526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:33.626526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:33.626526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:33.626526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:33.626526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:33.626526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:33.626526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:33.626526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:33.626526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:33.626526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:33.626526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:33.626526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:33.626526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:33.628202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:33.628202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:33.628202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:33.628202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:33.628202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.628202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:33.829542 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:33.830003 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:33.830263 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:33.831390 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:33.831595 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:33.831737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.832401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:33.832401 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:33.832401 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:33.832401 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:33.832401 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:33.832401 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:33.832401 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.832401 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:33.834698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.834698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:33.834698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:33.834698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:33.834698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.834698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:24:33.834698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:33.834698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:33.834698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:33.834698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:33.834698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:33.834698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:33.834945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:33.834945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:33.834945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:33.834945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:33.834945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:33.834945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:33.834945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:33.834945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:33.834945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:33.834945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:33.834945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:33.834945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:33.834945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:33.835184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.835184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:33.835230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.835230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:33.835230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:33.835230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:33.835230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:33.835230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:33.835230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.835230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:33.835230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:33.835230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.835230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:33.835230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:33.835230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.835230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:33.835230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:33.835230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.835519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:24:33.835519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:33.835519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:33.835519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:33.835519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:33.835519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:33.835519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:33.835519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:33.835519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:33.835519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:33.835519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:33.835519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:33.835519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:33.835519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:33.835519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:33.835519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:33.835519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:33.835519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:33.835519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:33.835898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:33.835898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.835898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:33.887149 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:33.887564 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:33.887802 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:33.888006 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:24:33.888622 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:33.888870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.889621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:33.889621 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:33.889621 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:33.889621 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:33.889621 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:33.889621 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:33.889621 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.889621 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:33.891268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.891268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:24:33.891268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:33.891268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:33.891268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:33.891268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:33.891268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:33.891268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:33.891268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:33.891268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:33.891268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:33.891268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:24:33.891268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:33.891268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:24:33.891268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:24:33.891619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:33.891619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:24:33.891619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:33.891619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:24:33.891619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:24:33.891619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:33.891619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:24:33.891619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.891619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:24:33.893211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.893211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:33.893211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:33.893211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:33.893211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.893211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:24:33.893211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:24:33.893211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:24:33.893211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:33.893211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:33.893211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:33.893211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:33.893562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:33.893562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:33.893562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:33.893562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:33.893562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:33.893562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:33.893562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:33.893562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:33.893562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:33.893562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:33.893562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:33.893562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:33.893562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:33.893562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:33.894357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:33.894357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:33.894357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.894357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:33.895154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.895154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:33.895154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:33.895154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:33.895154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:33.895154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:33.895154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.895154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:33.895154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:33.895154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.895154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:33.895154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:33.895154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.895154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:33.895154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:33.895154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.895526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:24:33.895526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:24:33.895526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:24:33.895526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:33.895526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:33.895526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:33.895526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:33.895526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:33.895526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:33.895526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:33.895526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:33.895526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:33.895526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:33.895526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:33.895526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:33.895526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:33.895526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:33.895526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:33.895954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:33.895954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:33.895954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:33.895954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:33.895954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:33.895954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.895954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:33.903471 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:33.903788 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:33.903910 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:33.904047 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:33.904943 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:33.905133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.905616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:33.905616 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:33.905616 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:33.905616 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:33.905616 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:33.905616 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:33.905616 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.905616 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:33.906829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.906829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:24:33.906829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:33.906829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:33.906829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:33.906829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:33.906829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:33.906829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:33.906829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:33.906829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:33.906829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:33.906829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:24:33.906829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:33.906829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:24:33.906829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:24:33.907300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:33.907300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:24:33.907300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:33.907300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:24:33.907300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:24:33.907300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:33.907300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:24:33.907300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.907300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:24:33.908199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.908199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:33.908199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:33.908199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:33.908199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.908199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:24:33.908199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:24:33.908199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:33.908199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:33.908199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:33.908199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:33.908580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:33.908580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:33.908580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:33.908580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:33.908580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:33.908580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:33.908580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:33.908580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:33.908580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:33.908580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:33.908580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:33.908580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:33.908580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:33.908580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:33.909054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:33.909054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:33.909054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.909054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:33.909947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.909947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:33.909947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:33.909947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:33.909947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:33.909947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:33.909947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.909947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:33.909947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:33.909947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.909947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:33.909947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:33.909947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.909947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:33.909947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:33.909947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.909947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:24:33.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:24:33.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:33.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:33.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:33.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:33.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:33.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:33.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:33.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:33.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:33.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:33.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:33.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:33.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:33.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:33.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:33.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:33.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:33.910399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:33.910915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:33.910915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:33.910915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.910915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:33.917419 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:33.917732 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:33.917840 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:33.918118 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:33.918465 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:33.918656 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:33.918779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.919262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:33.919262 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:33.919262 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:33.919262 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:33.919262 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:33.919262 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:33.919262 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.919262 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:33.920250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.920250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:33.920250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:33.920250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:33.920250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.920250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:24:33.920250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:33.920250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:33.920250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:33.920250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:33.920250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:33.920250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:33.920578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:33.920578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:33.920578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:33.920578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:33.920578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:33.920578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:33.920578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:33.920578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:33.920578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:33.920578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:33.920578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:33.920578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:33.920578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:33.920922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.920922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:33.921267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.921267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:33.921267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:33.921267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:33.921267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:33.921267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:33.921267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.921267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:33.921267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:33.921267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.921267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:33.921267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:33.921267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.921267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:33.921267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:33.921267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:33.921719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:24:33.921719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:33.921719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:33.921719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:33.921719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:33.921719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:33.921719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:33.921719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:33.921719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:33.921719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:33.921719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:33.921719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:33.921719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:33.921719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:33.921719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:33.921719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:33.921719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:33.921719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:33.921719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:33.922188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:33.922188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:33.922188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:38.595713 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:38.596451 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:38.596451 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:38.597644 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:38.632951 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:38.633338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.634410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:38.634410 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:38.634410 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:38.634410 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:38.634410 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:38.634410 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:38.634410 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.634410 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:38.635145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.635145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:38.635145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:38.635145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:38.635145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.635145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:24:38.635145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:38.635145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:38.635145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:38.635145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:38.635145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:38.635145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:38.635795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:38.635795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:38.635795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:38.635795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:38.635795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:38.635795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:38.635795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:38.635795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:38.635795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:38.635795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:38.635795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:38.635795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:38.635795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:38.636398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.636398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:38.636398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.636398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:38.636398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:38.636398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:38.636398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:38.636398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:38.636398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.636398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:38.636398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:38.636398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.636398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:38.636398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:38.636398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.637186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:38.637186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:38.637186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.637186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:24:38.637186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:38.637186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:38.637186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:38.637186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:38.637186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:38.637186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:38.637186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:38.637186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:38.637186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:38.637186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:38.637186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:38.637186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:38.637186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:38.637186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:38.637186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:38.638233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:38.638233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:38.638233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:38.638233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:38.638233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.638233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:38.830021 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:38.830573 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:38.830823 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:38.833657 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:38.834279 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:38.834877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.835970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:38.835970 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:38.835970 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:38.835970 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:38.835970 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:38.835970 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:38.835970 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.835970 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:38.837176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.837176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:38.837176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:38.837176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:38.837176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.837176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:24:38.837176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:38.837176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:38.837176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:38.837176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:38.837176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:38.837176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:38.837511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:38.837511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:38.837511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:38.837511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:38.837511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:38.837511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:38.837511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:38.837511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:38.837511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:38.837511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:38.837511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:38.837511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:38.837511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:38.837844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.837844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:38.838659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.838659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:38.838659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:38.838659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:38.838659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:38.838659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:38.838659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.838659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:38.838659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:38.838659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.838659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:38.838659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:38.838659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.838659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:38.838659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:38.838659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.839067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:24:38.839067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:38.839067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:38.839067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:38.839067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:38.839067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:38.839067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:38.839067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:38.839067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:38.839067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:38.839067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:38.839067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:38.839067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:38.839067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:38.839067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:38.839067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:38.839067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:38.839067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:38.839067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:38.839499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:38.839499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.839499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:38.887717 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:38.888194 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:38.888246 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:38.888370 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:24:38.888953 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:38.889183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.889770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:38.889770 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:38.889770 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:38.889770 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:38.889770 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:38.889770 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:38.889770 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.889770 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:38.890641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.890641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:24:38.890641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:38.890641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:38.890641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:38.890641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:38.890641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:38.890641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:38.890641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:38.890641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:38.890641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:38.890641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:24:38.890641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:38.890641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:24:38.890641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:24:38.891079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:38.891079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:24:38.891079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:38.891079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:24:38.891079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:24:38.891079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:38.891079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:24:38.891079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.891079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:24:38.891544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.891544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:38.891544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:38.891544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:38.891544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.891544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:24:38.891544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:24:38.891544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:24:38.891544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:38.891544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:38.891544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:38.891544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:38.891934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:38.891934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:38.891934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:38.891934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:38.891934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:38.891934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:38.891934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:38.891934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:38.891934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:38.891934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:38.891934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:38.891934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:38.891934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:38.891934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:38.892273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:38.892273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:38.892273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.892273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:38.892699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.892699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:38.892699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:38.892699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:38.892699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:38.892699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:38.892699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.892699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:38.892699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:38.892699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.892699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:38.892699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:38.892699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.892699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:38.892699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:38.892699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.893153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:24:38.893153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:24:38.893153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:24:38.893153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:38.893153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:38.893153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:38.893153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:38.893153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:38.893153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:38.893153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:38.893153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:38.893153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:38.893153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:38.893153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:38.893153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:38.893153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:38.893153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:38.893153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:38.893563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:38.893563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:38.893563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:38.893563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:38.893563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:38.893563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.893563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:38.904046 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:38.904804 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:38.904928 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:38.905068 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:38.905473 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:38.905754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.906329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:38.906329 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:38.906329 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:38.906329 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:38.906329 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:38.906329 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:38.906329 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.906329 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:38.907261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.907261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:24:38.907261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:38.907261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:38.907261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:38.907261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:38.907261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:38.907261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:38.907261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:38.907261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:38.907261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:38.907261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:24:38.907261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:38.907261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:24:38.907261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:24:38.907599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:38.907599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:24:38.907599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:38.907599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:24:38.907599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:24:38.907599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:38.907599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:24:38.907599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.907599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:24:38.908257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.908257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:38.908257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:38.908257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:38.908257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.908257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:24:38.908257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:24:38.908257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:38.908257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:38.908257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:38.908257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:38.908579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:38.908579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:38.908579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:38.908579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:38.908579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:38.908579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:38.908579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:38.908579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:38.908579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:38.908579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:38.908579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:38.908579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:38.908579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:38.908579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:38.908950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:38.908950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:38.908950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.908950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:38.909402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.909402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:38.909402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:38.909402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:38.909402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:38.909402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:38.909402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.909402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:38.909402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:38.909402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.909402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:38.909402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:38.909402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.909402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:38.909402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:38.909402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.909402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:24:38.909869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:24:38.909869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:38.909869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:38.909869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:38.909869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:38.909869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:38.909869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:38.909869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:38.909869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:38.909869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:38.909869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:38.909869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:38.909869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:38.909869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:38.909869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:38.909869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:38.909869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:38.909869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:38.909869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:38.911055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:38.911055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:38.911055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.911055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:38.918195 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:38.918389 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:38.918503 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:38.918704 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:38.919273 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:38.919699 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:38.919999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.920827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:38.920827 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:38.920827 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:38.920827 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:38.920827 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:38.920827 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:38.920827 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.920827 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:38.921847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.921847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:38.921847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:38.921847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:38.921847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.921847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:24:38.921847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:38.921847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:38.921847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:38.921847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:38.921847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:38.921847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:38.922204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:38.922204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:38.922204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:38.922204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:38.922204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:38.922204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:38.922204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:38.922204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:38.922204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:38.922204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:38.922204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:38.922204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:38.922204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:38.922513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.922513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:38.923065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.923065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:38.923065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:38.923065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:38.923065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:38.923065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:38.923065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.923065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:38.923065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:38.923065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.923065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:38.923065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:38.923065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.923065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:38.923065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:38.923065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:38.923529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:24:38.923529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:38.923529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:38.923529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:38.923529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:38.923529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:38.923529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:38.923529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:38.923529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:38.923529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:38.923529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:38.923529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:38.923529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:38.923529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:38.923529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:38.923529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:38.923529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:38.923529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:38.923529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:38.923988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:38.923988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:38.923988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:43.596333 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:43.596989 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:43.597309 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:43.598568 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:43.627926 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:43.628297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.629294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:43.629294 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:43.629294 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:43.629294 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:43.629294 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:43.629294 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:43.629294 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.629294 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:43.630060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.630060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:43.630060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:43.630060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:43.630060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.630060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:24:43.630060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:43.630060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:43.630060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:43.630060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:43.630060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:43.630060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:43.630803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:43.630803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:43.630803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:43.630803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:43.630803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:43.630803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:43.630803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:43.630803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:43.630803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:43.630803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:43.630803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:43.630803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:43.630803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:43.631619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.631619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:43.631619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.631619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:43.631619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:43.631619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:43.631619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:43.631619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:43.631619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.631619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:43.631619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:43.631619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.631619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:43.631619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:43.631619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.632537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:43.632537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:43.632537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.632537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:24:43.632537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:43.632537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:43.632537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:43.632537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:43.632537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:43.632537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:43.632537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:43.632537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:43.632537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:43.632537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:43.632537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:43.632537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:43.632537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:43.632537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:43.632537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:43.633538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:43.633538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:43.633538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:43.633538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:43.633538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.633538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:43.830750 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:43.831109 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:43.831299 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:43.832573 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:43.832573 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:43.832573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.832836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:43.832836 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:43.832836 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:43.832836 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:43.832836 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:43.832836 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:43.832836 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.832836 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:43.834352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.834352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:43.834352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:43.834352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:43.834352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.834352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:24:43.834352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:43.834352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:43.834352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:43.834352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:43.834352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:43.834352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:43.834802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:43.834802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:43.834802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:43.834802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:43.834802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:43.834802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:43.834802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:43.834802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:43.834802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:43.834802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:43.834802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:43.834802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:43.834802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:43.835160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.835160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:43.836416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.836416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:43.836416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:43.836416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:43.836416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:43.836416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:43.836416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.836416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:43.836416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:43.836416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.836416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:43.836416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:43.836416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.836416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:43.836416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:43.836416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.836931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:24:43.836931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:43.836931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:43.836931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:43.836931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:43.836931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:43.836931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:43.836931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:43.836931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:43.836931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:43.836931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:43.836931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:43.836931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:43.836931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:43.836931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:43.836931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:43.836931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:43.836931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:43.836931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:43.837449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:43.837449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.837449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:43.888470 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:43.888932 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:43.889004 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:43.889150 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:24:43.889958 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:43.890637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.891083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:43.891083 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:43.891083 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:43.891083 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:43.891083 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:43.891083 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:43.891083 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.891083 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:43.891850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.891850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:24:43.891850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:43.891850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:43.891850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:43.891850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:43.891850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:43.891850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:43.891850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:43.891850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:43.891850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:43.891850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:24:43.891850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:43.891850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:24:43.891850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:24:43.892320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:43.892320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:24:43.892320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:43.892320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:24:43.892320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:24:43.892320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:43.892320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:24:43.892320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.892320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:24:43.892754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.892754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:43.892754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:43.892754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:43.892754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.892754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:24:43.892754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:24:43.892754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:24:43.892754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:43.892754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:43.892754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:43.892754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:43.893117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:43.893117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:43.893117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:43.893117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:43.893117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:43.893117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:43.893117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:43.893117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:43.893117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:43.893117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:43.893117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:43.893117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:43.893117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:43.893117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:43.893472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:43.893472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:43.893472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.893472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:43.894046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.894046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:43.894046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:43.894046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:43.894046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:43.894046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:43.894046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.894046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:43.894046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:43.894046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.894046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:43.894046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:43.894046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.894046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:43.894046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:43.894046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.894497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:24:43.894497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:24:43.894497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:24:43.894497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:43.894497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:43.894497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:43.894497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:43.894497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:43.894497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:43.894497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:43.894497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:43.894497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:43.894497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:43.894497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:43.894497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:43.894497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:43.894497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:43.894497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:43.895449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:43.895449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:43.895449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:43.895449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:43.895449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:43.895449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.895449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:43.904976 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:43.905826 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:43.905826 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:43.905826 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:43.906455 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:43.906988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.907911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:43.907911 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:43.907911 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:43.907911 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:43.907911 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:43.907911 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:43.907911 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.907911 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:43.909157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.909157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:24:43.909157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:43.909157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:43.909157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:43.909157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:43.909157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:43.909157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:43.909157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:43.909157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:43.909157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:43.909157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:24:43.909157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:43.909157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:24:43.909157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:24:43.909942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:43.909942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:24:43.909942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:43.909942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:24:43.909942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:24:43.909942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:43.909942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:24:43.909942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.909942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:24:43.910600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.910600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:43.910600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:43.910600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:43.910600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.910600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:24:43.910600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:24:43.910600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:43.910600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:43.910600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:43.910600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:43.911211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:43.911211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:43.911211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:43.911211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:43.911211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:43.911211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:43.911211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:43.911211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:43.911211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:43.911211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:43.911211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:43.911211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:43.911211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:43.911211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:43.911825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:43.911825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:43.911825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.911825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:43.912483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.912483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:43.912483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:43.912483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:43.912483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:43.912483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:43.912483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.912483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:43.912483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:43.912483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.912483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:43.912483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:43.912483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.912483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:43.912483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:43.912483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.912483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:24:43.913346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:24:43.913346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:43.913346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:43.913346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:43.913346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:43.913346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:43.913346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:43.913346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:43.913346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:43.913346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:43.913346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:43.913346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:43.913346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:43.913346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:43.913346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:43.913346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:43.913346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:43.913346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:43.913346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:43.914448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:43.914448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:43.914448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.914448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:43.918394 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:43.918739 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:43.918813 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:43.918989 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:43.919460 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:43.919870 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:43.920122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.920794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:43.920794 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:43.920794 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:43.920794 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:43.920794 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:43.920794 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:43.920794 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.920794 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:43.921608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.921608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:43.921608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:43.921608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:43.921608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.921608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:24:43.921608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:43.921608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:43.921608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:43.921608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:43.921608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:43.921608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:43.922009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:43.922009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:43.922009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:43.922009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:43.922009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:43.922009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:43.922009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:43.922009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:43.922009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:43.922009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:43.922009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:43.922009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:43.922009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:43.922470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.922470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:43.923036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.923036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:43.923036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:43.923036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:43.923036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:43.923036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:43.923036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.923036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:43.923036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:43.923036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.923036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:43.923036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:43.923036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.923036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:43.923036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:43.923036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:43.923774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:24:43.923774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:43.923774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:43.923774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:43.923774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:43.923774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:43.923774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:43.923774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:43.923774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:43.923774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:43.923774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:43.923774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:43.923774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:43.923774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:43.923774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:43.923774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:43.923774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:43.923774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:43.923774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:43.924326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:43.924326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:43.924326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:48.597163 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:48.597910 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:48.597910 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:48.598597 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:48.635573 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:48.635573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.636283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:48.636283 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:48.636283 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:48.636283 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:48.636283 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:48.636283 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:48.636283 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.636283 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:48.637546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.637546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:48.637546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:48.637546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:48.637546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.637546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:24:48.637546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:48.637546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:48.637546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:48.637546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:48.637546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:48.637546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:48.637904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:48.637904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:48.637904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:48.637904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:48.637904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:48.637904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:48.637904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:48.637904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:48.637904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:48.637904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:48.637904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:48.637904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:48.637904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:48.638233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.638233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:48.638366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.638366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:48.638366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:48.638366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:48.638366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:48.638366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:48.638366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.638366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:48.638366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:48.638366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.638366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:48.638366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:48.638366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.638366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:48.638366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:48.638366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.638755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:24:48.638755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:48.638755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:48.638755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:48.638755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:48.638755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:48.638755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:48.638755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:48.638755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:48.638755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:48.638755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:48.638755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:48.638755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:48.638755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:48.638755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:48.638755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:48.638755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:48.638755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:48.638755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:48.639156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:48.639156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.639156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:48.831515 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:48.832201 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:48.832294 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:48.833218 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:48.833479 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:48.833619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.834339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:48.834339 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:48.834339 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:48.834339 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:48.834339 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:48.834339 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:48.834339 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.834339 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:48.835750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.835750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:48.835750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:48.835750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:48.835750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.835750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:24:48.835750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:48.835750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:48.835750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:48.835750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:48.835750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:48.835750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:48.836372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:48.836372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:48.836372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:48.836372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:48.836372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:48.836372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:48.836372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:48.836372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:48.836372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:48.836372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:48.836372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:48.836372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:48.836372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:48.837043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.837043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:48.837158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.837158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:48.837158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:48.837158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:48.837158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:48.837158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:48.837158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.837158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:48.837158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:48.837158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.837158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:48.837158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:48.837158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.837158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:48.837158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:48.837158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.838012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:24:48.838012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:48.838012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:48.838012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:48.838012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:48.838012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:48.838012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:48.838012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:48.838012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:48.838012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:48.838012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:48.838012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:48.838012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:48.838012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:48.838012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:48.838012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:48.838012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:48.838012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:48.838012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:48.839071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:48.839071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.839071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:48.888969 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:48.889435 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:48.889548 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:48.889702 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:24:48.890335 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:48.890458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.890940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:48.890940 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:48.890940 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:48.890940 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:48.890940 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:48.890940 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:48.890940 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.890940 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:48.891787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.891787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:24:48.891787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:48.891787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:48.891787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:48.891787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:48.891787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:48.891787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:48.891787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:48.891787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:48.891787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:48.891787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:24:48.891787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:48.891787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:24:48.891787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:24:48.892511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:48.892511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:24:48.892511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:48.892511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:24:48.892511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:24:48.892511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:48.892511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:24:48.892511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.892511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:24:48.892996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.892996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:48.892996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:48.892996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:48.892996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.892996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:24:48.892996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:24:48.892996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:24:48.892996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:48.892996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:48.892996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:48.892996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:48.893537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:48.893537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:48.893537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:48.893537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:48.893537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:48.893537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:48.893537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:48.893537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:48.893537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:48.893537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:48.893537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:48.893537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:48.893537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:48.893537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:48.894186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:48.894186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:48.894186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.894186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:48.894186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.894186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:48.894186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:48.894186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:48.894186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:48.894186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:48.894186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.894186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:48.894186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:48.894186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.894900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:48.894900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:48.894900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.894900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:48.894900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:48.894900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.894900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:24:48.894900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:24:48.894900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:24:48.894900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:48.894900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:48.894900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:48.894900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:48.894900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:48.894900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:48.894900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:48.894900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:48.894900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:48.894900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:48.895899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:48.895899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:48.895899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:48.895899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:48.895899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:48.895899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:48.895899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:48.895899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:48.895899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:48.895899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:48.895899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.895899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:48.905541 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:48.906064 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:48.906256 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:48.906500 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:48.907097 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:48.907486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.908310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:48.908310 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:48.908310 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:48.908310 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:48.908310 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:48.908310 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:48.908310 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.908310 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:48.909648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.909648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:24:48.909648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:48.909648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:48.909648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:48.909648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:48.909648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:48.909648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:48.909648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:48.909648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:48.909648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:48.909648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:24:48.909648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:48.909648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:24:48.909648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:24:48.910350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:48.910350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:24:48.910350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:48.910350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:24:48.910350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:24:48.910350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:48.910350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:24:48.910350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.910350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:24:48.911198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.911198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:48.911198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:48.911198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:48.911198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.911198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:24:48.911198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:24:48.911198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:48.911198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:48.911198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:48.911198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:48.911910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:48.911910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:48.911910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:48.911910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:48.911910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:48.911910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:48.911910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:48.911910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:48.911910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:48.911910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:48.911910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:48.911910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:48.911910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:48.911910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:48.912597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:48.912597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:48.912597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.912597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:48.913054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.913054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:48.913054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:48.913054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:48.913054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:48.913054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:48.913054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.913054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:48.913054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:48.913054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.913054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:48.913054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:48.913054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.913054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:48.913054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:48.913054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.913054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:24:48.913984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:24:48.913984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:48.913984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:48.913984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:48.913984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:48.913984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:48.913984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:48.913984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:48.913984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:48.913984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:48.913984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:48.913984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:48.913984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:48.913984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:48.913984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:48.913984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:48.913984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:48.913984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:48.913984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:48.914529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:48.914529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:48.914529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.914529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:48.918869 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:48.919172 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:48.919288 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:48.919459 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:48.919931 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:48.920234 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:48.920636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.921484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:48.921484 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:48.921484 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:48.921484 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:48.921484 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:48.921484 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:48.921484 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.921484 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:48.922335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.922335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:48.922335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:48.922335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:48.922335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.922335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:24:48.922335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:48.922335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:48.922335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:48.922335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:48.922335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:48.922335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:48.922987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:48.922987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:48.922987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:48.922987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:48.922987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:48.922987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:48.922987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:48.922987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:48.922987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:48.922987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:48.922987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:48.922987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:48.922987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:48.923657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.923657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:48.923657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.923657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:48.923657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:48.923657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:48.923657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:48.923657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:48.923657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.923657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:48.923657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:48.923657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.923657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:48.923657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:48.923657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.924354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:48.924354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:48.924354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:48.924354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:24:48.924354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:48.924354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:48.924354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:48.924354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:48.924354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:48.924354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:48.924354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:48.924354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:48.924354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:48.924354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:48.924354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:48.924354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:48.924354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:48.924354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:48.924354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:48.925290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:48.925290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:48.925290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:48.925290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:48.925290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:48.925290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:53.598014 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:53.598595 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:53.598595 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:53.599344 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:53.630246 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:53.630246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.630246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:53.630246 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:53.630246 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:53.630246 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:53.630246 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:53.630246 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:53.630246 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.630246 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:53.630246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.630246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:53.630246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:53.630836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:53.630836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.630836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:24:53.630836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:53.630836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:53.630836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:53.630836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:53.630836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:53.630836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:53.630836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:53.630836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:53.630836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:53.630836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:53.630836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:53.631300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:53.631300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:53.631300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:53.631300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:53.631300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:53.631300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:53.631300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:53.631300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:53.631300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.631300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:53.631300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.631300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:53.631697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:53.631697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:53.631697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:53.631697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:53.631697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.631697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:53.631697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:53.631697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.631697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:53.631697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:53.631697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.631697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:53.631697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:53.631697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.631697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:24:53.631697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:53.631697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:53.631697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:53.631697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:53.632357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:53.632357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:53.632357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:53.632357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:53.632357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:53.632357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:53.632357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:53.632357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:53.632357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:53.632357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:53.632357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:53.632357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:53.632357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:53.632357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:53.632357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:53.632357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.632357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:53.832390 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:53.832862 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:53.833046 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:53.833960 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:53.835239 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:53.835239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.835239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:53.835239 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:53.835239 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:53.835239 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:53.835239 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:53.835239 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:53.835239 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.835239 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:53.836563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.836563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:53.836563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:53.836563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:53.836563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.836563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:24:53.836563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:53.836563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:53.836563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:53.836563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:53.836563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:53.836563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:53.837361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:53.837361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:53.837361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:53.837361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:53.837361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:53.837361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:53.837361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:53.837361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:53.837361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:53.837361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:53.837361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:53.837361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:53.837361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:53.837737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.837737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:53.837737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.837737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:53.837737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:53.837737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:53.837737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:53.837737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:53.837737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.837737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:53.837737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:53.837737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.837737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:53.837737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:53.837737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.838175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:53.838175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:53.838175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.838175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:24:53.838175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:53.838175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:53.838175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:53.838175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:53.838175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:53.838175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:53.838175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:53.838175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:53.838175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:53.838175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:53.838175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:53.838175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:53.838175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:53.838175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:53.838175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:53.838712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:53.838712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:53.838712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:53.838712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:53.838712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.838712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:53.889502 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:53.889948 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:53.890063 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:53.890186 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:24:53.890697 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:53.891946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.891946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:53.891946 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:53.891946 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:53.891946 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:53.891946 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:53.891946 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:53.891946 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.891946 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:53.892279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.892279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:24:53.892279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:53.892279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:53.892279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:53.892279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:53.892279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:53.892279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:53.892279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:53.892279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:53.892279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:53.892279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:24:53.892279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:53.892279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:24:53.892279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:24:53.892905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:53.892905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:24:53.892905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:53.892905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:24:53.892905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:24:53.892905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:53.892905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:24:53.892905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.892905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:24:53.896519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.896519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:53.896519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:53.896519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:53.896519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.896519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:24:53.896519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:24:53.896519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:24:53.896519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:53.896519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:53.896519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:53.896519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:53.896881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:53.896881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:53.896881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:53.896881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:53.896881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:53.896881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:53.896881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:53.896881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:53.896881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:53.896881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:53.896881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:53.896881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:53.896881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:53.896881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:53.897185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:53.897185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:53.897185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.897185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:53.897185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.897185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:53.897185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:53.897185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:53.897185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:53.897185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:53.897185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.897185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:53.897185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:53.897185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.897475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:53.897475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:53.897475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.897475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:53.897475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:53.897475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.897475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:24:53.897475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:24:53.897475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:24:53.897475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:53.897475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:53.897475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:53.897475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:53.897475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:53.897475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:53.897475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:53.897475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:53.897475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:53.897475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:53.897881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:53.897881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:53.897881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:53.897881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:53.897881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:53.897881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:53.897881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:53.897881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:53.897881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:53.897881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:53.897881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.897881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:53.906068 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:53.906356 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:53.906469 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:53.906593 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:53.907648 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:53.907648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.907849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:53.907849 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:53.907849 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:53.907849 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:53.907849 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:53.907849 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:53.907849 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.907849 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:53.908555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.908555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:24:53.908555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:53.908555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:53.908555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:53.908555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:53.908555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:53.908555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:53.908555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:53.908555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:53.908555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:53.908555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:24:53.908555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:53.908555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:24:53.908555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:24:53.909245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:53.909245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:24:53.909245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:53.909245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:24:53.909245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:24:53.909245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:53.909245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:24:53.909245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.909245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:24:53.909983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.909983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:53.909983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:53.909983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:53.909983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.909983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:24:53.909983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:24:53.909983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:53.909983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:53.909983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:53.909983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:53.910617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:53.910617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:53.910617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:53.910617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:53.910617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:53.910617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:53.910617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:53.910617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:53.910617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:53.910617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:53.910617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:53.910617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:53.910617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:53.910617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:53.911299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:53.911299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:53.911299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.911299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:53.911299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.911299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:53.911299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:53.911299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:53.911299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:53.911299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:53.911299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.911299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:53.911299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:53.911299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.911921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:53.911921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:53.911921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.911921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:53.911921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:53.911921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.911921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:24:53.911921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:24:53.911921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:53.911921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:53.911921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:53.911921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:53.911921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:53.911921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:53.911921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:53.911921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:53.911921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:53.911921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:53.911921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:53.912738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:53.912738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:53.912738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:53.912738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:53.912738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:53.912738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:53.912738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:53.912738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:53.912738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:53.912738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.912738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:53.919333 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:53.919745 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:53.919867 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:53.920050 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:53.920638 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:53.921026 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:53.921297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.922072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:53.922072 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:53.922072 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:53.922072 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:53.922072 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:53.922072 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:53.922072 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.922072 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:53.922782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.922782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:53.922782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:53.922782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:53.922782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.922782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:24:53.922782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:53.922782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:53.922782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:53.922782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:53.922782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:53.922782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:53.923168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:53.923168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:53.923168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:53.923168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:53.923168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:53.923168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:53.923168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:53.923168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:53.923168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:53.923168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:53.923168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:53.923168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:53.923168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:53.923503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.923503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:53.923503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.923503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:53.923503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:53.923503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:53.923503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:53.923503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:53.923503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.923503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:53.923503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:53.923503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.923503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:53.923503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:53.923503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.923928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:53.923928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:53.923928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:53.923928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:24:53.923928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:53.923928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:53.923928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:53.923928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:53.923928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:53.923928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:53.923928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:53.923928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:53.923928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:53.923928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:53.923928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:53.923928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:53.923928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:53.923928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:53.923928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:53.924437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:53.924437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:53.924437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:53.924437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:53.924437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:53.924437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:58.598595 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:58.599452 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:58.599452 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:58.599980 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:58.631801 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:58.632031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.632807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:58.632807 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:58.632807 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:58.632807 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:58.632807 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:58.632807 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:58.632807 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.632807 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:58.633508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.633508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:58.633508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:58.633508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:58.633508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.633508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:24:58.633508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:58.633508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:58.633508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:58.633508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:58.633508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:58.633508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:58.633862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:58.633862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:58.633862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:58.633862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:58.633862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:58.633862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:58.633862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:58.633862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:58.633862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:58.633862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:58.633862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:58.633862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:58.633862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:58.634183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.634183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:58.634242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.634242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:58.634242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:58.634242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:58.634242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:58.634242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:58.634242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.634242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:58.634242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:58.634242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.634242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:58.634242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:58.634242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.634242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:58.634242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:58.634242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.634627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:24:58.634627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:58.634627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:58.634627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:58.634627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:58.634627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:58.634627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:58.634627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:58.634627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:58.634627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:58.634627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:58.634627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:58.634627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:58.634627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:58.634627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:58.634627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:58.634627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:58.634627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:58.634627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:58.635103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:58.635103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.635103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:58.833047 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:58.833483 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:58.833696 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:58.834533 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:58.837790 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:58.837790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.837790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:58.837790 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:58.837790 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:58.837790 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:58.837790 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:58.837790 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:58.837790 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.837790 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:58.837790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.837790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:58.837790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:58.837790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:58.839062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.839062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:24:58.839062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:58.839062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:58.839062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:58.839062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:58.839062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:58.839062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:58.839062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:58.839062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:58.839062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:58.839062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:58.839062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:58.839062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:58.839720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:58.839720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:58.839720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:58.839720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:58.839720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:58.839720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:58.839720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:58.839720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.839720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:58.839720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.839720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:58.839720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:58.840136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:58.840136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:58.840136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:58.840136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.840136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:58.840136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:58.840136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.840136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:58.840136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:58.840136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.840136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:58.840136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:58.840136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.840136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:24:58.840136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:58.840136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:58.840136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:58.840136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:58.840136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:58.840136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:58.840776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:58.840776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:58.840776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:58.840776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:58.840776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:58.840776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:58.840776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:58.840776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:58.840776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:58.840776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:58.840776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:58.840776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:58.840776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:58.840776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.840776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:58.890139 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:58.890537 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:58.890739 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:58.890925 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:24:58.891727 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:58.892096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.892971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:58.892971 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:58.892971 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:58.892971 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:58.892971 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:58.892971 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:58.892971 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.892971 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:58.894119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.894119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:24:58.894119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:58.894119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:58.894119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:58.894119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:58.894119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:58.894119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:58.894119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:58.894119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:58.894119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:58.894119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:24:58.894119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:58.894119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:24:58.894119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:24:58.894817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:58.894817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:24:58.894817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:58.894817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:24:58.894817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:24:58.894817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:58.894817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:24:58.894817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.894817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:24:58.895404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.895404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:58.895404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:58.895404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:58.895404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.895404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:24:58.895404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:24:58.895404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:24:58.895404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:58.895404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:58.895404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:58.895404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:58.895930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:58.895930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:58.895930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:58.895930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:58.895930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:58.895930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:58.895930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:58.895930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:58.895930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:58.895930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:58.895930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:58.895930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:58.895930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:58.895930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:58.896328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:58.896328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:58.896328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.896328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:58.896982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.896982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:58.896982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:58.896982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:58.896982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:58.896982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:58.896982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.896982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:58.896982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:58.896982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.896982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:58.896982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:58.896982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.896982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:58.896982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:58.896982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.897705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:24:58.897705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:24:58.897705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:24:58.897705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:58.897705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:58.897705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:58.897705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:58.897705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:58.897705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:58.897705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:58.897705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:58.897705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:58.897705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:58.897705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:58.897705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:58.897705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:58.897705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:58.897705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:58.898496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:58.898496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:58.898496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:58.898496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:58.898496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:58.898496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.898496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:58.906586 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:58.906877 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:58.907003 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:58.907139 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:58.907550 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:58.907944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.908695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:58.908695 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:58.908695 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:58.908695 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:58.908695 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:58.908695 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:58.908695 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.908695 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:58.909826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.909826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:24:58.909826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:58.909826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:58.909826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:58.909826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:58.909826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:58.909826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:58.909826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:58.909826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:24:58.909826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:58.909826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:24:58.909826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:58.909826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:24:58.909826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:24:58.910330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:58.910330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:24:58.910330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:58.910330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:24:58.910330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:24:58.910330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:58.910330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:24:58.910330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.910330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:24:58.911196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.911196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:58.911196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:58.911196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:58.911196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.911196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:24:58.911196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:24:58.911196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:58.911196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:58.911196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:58.911196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:58.911838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:58.911838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:58.911838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:58.911838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:58.911838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:58.911838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:58.911838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:58.911838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:58.911838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:58.911838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:58.911838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:58.911838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:58.911838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:58.911838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:58.912333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:58.912333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:58.912333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.912333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:58.912835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.912835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:58.912835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:58.912835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:58.912835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:58.912835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:58.912835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.912835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:58.912835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:58.912835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.912835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:58.912835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:58.912835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.912835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:58.912835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:58.912835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.912835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:24:58.913272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:24:58.913272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:24:58.913272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:24:58.913272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:58.913272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:58.913272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:58.913272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:58.913272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:58.913272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:58.913272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:58.913272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:58.913272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:58.913272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:58.913272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:58.913272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:58.913272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:58.913272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:58.913272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:58.913272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:58.913634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:58.913634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:58.913634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.913634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:24:58.919795 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:24:58.920067 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:24:58.920184 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:24:58.920337 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:24:58.920687 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:58.920985 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:24:58.921177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.921684 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:24:58.921684 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:58.921684 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:24:58.921684 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:24:58.921684 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:58.921684 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:24:58.921684 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.921684 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:24:58.922546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.922546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:24:58.922546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:58.922546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:24:58.922546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.922546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:24:58.922546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:58.922546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:58.922546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:58.922546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:58.922546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:58.922546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:58.922967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:24:58.922967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:58.922967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:24:58.922967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:58.922967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:24:58.922967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:24:58.922967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:58.922967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:24:58.922967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:58.922967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:24:58.922967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:24:58.922967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:58.922967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:24:58.923326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.923326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:24:58.923699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.923699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:24:58.923699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:24:58.923699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:24:58.923699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:24:58.923699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:24:58.923699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.923699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:24:58.923699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:24:58.923699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.923699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:24:58.923699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:24:58.923699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.923699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:24:58.923699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:24:58.923699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:24:58.924175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:24:58.924175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:24:58.924175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:58.924175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:58.924175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:24:58.924175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:24:58.924175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:24:58.924175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:24:58.924175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:24:58.924175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:24:58.924175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:24:58.924175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:24:58.924175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:24:58.924175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:24:58.924175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:24:58.924175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:24:58.924175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:24:58.924175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:24:58.924175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:24:58.924712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:24:58.924712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:24:58.924712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:03.599142 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:03.599864 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:03.599864 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:03.600269 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:03.601436 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=240 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:25:03.633174 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:03.633945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.634851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:03.634851 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:03.634851 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:03.634851 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:03.634851 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:03.634851 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:03.634851 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.634851 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:03.637316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.637316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:03.637316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:03.637316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:03.637316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.637316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:25:03.637316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:03.637316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:03.637316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:03.637316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:03.637316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:03.637316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:03.637854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:03.637854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:03.637854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:03.637854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:03.637854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:03.637854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:03.637854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:03.637854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:03.637854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:03.637854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:03.637854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:03.637854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:03.637854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:03.638301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.638301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:03.638301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.638301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:03.638301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:03.638301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:03.638301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:03.638301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:03.638301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.638301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:03.638301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:03.638301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.638301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:03.638301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:03.638301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.638708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:03.638708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:03.638708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.638708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:25:03.638708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:03.638708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:03.638708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:03.638708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:03.638708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:03.638708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:03.638708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:03.638708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:03.638708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:03.638708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:03.638708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:03.638708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:03.638708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:03.638708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:03.638708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:03.639170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:03.639170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:03.639170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:03.639170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:03.639170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.639170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:03.834178 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:03.834178 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:03.834589 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:03.835421 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:03.835699 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:03.835897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.836558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:03.836558 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:03.836558 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:03.836558 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:03.836558 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:03.836558 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:03.836558 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.836558 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:03.838219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.838219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:03.838219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:03.838219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:03.838219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.838219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:25:03.838219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:03.838219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:03.838219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:03.838219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:03.838219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:03.838219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:03.839376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:03.839376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:03.839376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:03.839376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:03.839376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:03.839376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:03.839376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:03.839376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:03.839376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:03.839376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:03.839376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:03.839376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:03.839376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:03.839786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.839786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:03.840370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.840370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:03.840370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:03.840370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:03.840370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:03.840370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:03.840370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.840370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:03.840370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:03.840370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.840370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:03.840370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:03.840370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.840370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:03.840370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:03.840370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.840939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:25:03.840939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:03.840939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:03.840939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:03.840939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:03.840939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:03.840939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:03.840939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:03.840939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:03.840939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:03.840939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:03.840939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:03.840939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:03.840939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:03.840939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:03.840939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:03.840939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:03.840939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:03.840939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:03.841464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:03.841464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.841464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:03.890784 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:03.891256 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:03.891438 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:03.891622 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:25:03.892275 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:03.892643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.893612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:03.893612 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:03.893612 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:03.893612 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:03.893612 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:03.893612 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:03.893612 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.893612 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:03.894892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.894892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:25:03.894892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:03.894892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:03.894892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:03.894892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:03.894892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:03.894892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:03.894892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:03.894892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:03.894892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:03.894892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:25:03.894892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:03.894892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:25:03.894892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:25:03.895315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:03.895315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:25:03.895315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:03.895315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:25:03.895315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:25:03.895315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:03.895315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:25:03.895315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.895315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:25:03.896236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.896236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:03.896236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:03.896236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:03.896236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.896236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:25:03.896236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:25:03.896236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:25:03.896236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:03.896236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:03.896236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:03.896236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:03.896570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:03.896570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:03.896570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:03.896570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:03.896570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:03.896570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:03.896570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:03.896570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:03.896570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:03.896570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:03.896570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:03.896570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:03.896570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:03.896570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:03.896987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:03.896987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:03.896987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.896987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:03.898023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.898023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:03.898023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:03.898023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:03.898023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:03.898023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:03.898023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.898023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:03.898023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:03.898023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.898023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:03.898023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:03.898023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.898023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:03.898023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:03.898023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.898432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:25:03.898432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:25:03.898432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:25:03.898432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:03.898432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:03.898432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:03.898432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:03.898432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:03.898432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:03.898432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:03.898432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:03.898432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:03.898432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:03.898432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:03.898432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:03.898432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:03.898432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:03.898432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:03.898902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:03.898902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:03.898902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:03.898902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:03.898902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:03.898902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.898902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:03.907372 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:03.907736 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:03.907815 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:03.907982 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:03.908483 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:03.908574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.909104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:03.909104 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:03.909104 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:03.909104 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:03.909104 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:03.909104 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:03.909104 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.909104 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:03.910054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.910054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:25:03.910054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:03.910054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:03.910054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:03.910054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:03.910054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:03.910054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:03.910054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:03.910054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:03.910054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:03.910054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:25:03.910054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:03.910054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:25:03.910054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:25:03.910434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:03.910434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:25:03.910434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:03.910434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:25:03.910434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:25:03.910434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:03.910434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:25:03.910434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.910434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:25:03.910978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.910978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:03.910978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:03.910978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:03.910978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.910978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:25:03.910978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:25:03.910978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:03.910978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:03.910978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:03.910978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:03.911226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:03.911226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:03.911226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:03.911226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:03.911226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:03.911226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:03.911226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:03.911226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:03.911226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:03.911226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:03.911226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:03.911226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:03.911226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:03.911226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:03.911478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:03.911478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:03.911478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.911478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:03.912179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.912179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:03.912179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:03.912179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:03.912179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:03.912179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:03.912179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.912179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:03.912179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:03.912179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.912179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:03.912179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:03.912179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.912179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:03.912179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:03.912179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.912179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:25:03.912617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:25:03.912617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:03.912617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:03.912617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:03.912617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:03.912617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:03.912617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:03.912617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:03.912617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:03.912617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:03.912617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:03.912617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:03.912617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:03.912617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:03.912617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:03.912617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:03.912617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:03.912617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:03.912617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:03.913105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:03.913105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:03.913105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.913105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:03.920254 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:03.920618 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:03.920961 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:03.921169 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:03.921741 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:03.922241 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:03.922486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.923330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:03.923330 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:03.923330 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:03.923330 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:03.923330 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:03.923330 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:03.923330 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.923330 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:03.924754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.924754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:03.924754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:03.924754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:03.924754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.924754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:25:03.924754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:03.924754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:03.924754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:03.924754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:03.924754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:03.924754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:03.925139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:03.925139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:03.925139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:03.925139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:03.925139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:03.925139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:03.925139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:03.925139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:03.925139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:03.925139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:03.925139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:03.925139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:03.925139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:03.925448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.925448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:03.926333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.926333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:03.926333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:03.926333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:03.926333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:03.926333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:03.926333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.926333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:03.926333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:03.926333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.926333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:03.926333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:03.926333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.926333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:03.926333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:03.926333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:03.926792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:25:03.926792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:03.926792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:03.926792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:03.926792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:03.926792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:03.926792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:03.926792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:03.926792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:03.926792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:03.926792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:03.926792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:03.926792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:03.926792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:03.926792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:03.926792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:03.926792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:03.926792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:03.926792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:03.927268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:03.927268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:03.927268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:08.599908 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:08.600741 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:08.600741 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:08.601250 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:08.635787 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:08.637716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.637716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:08.637716 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:08.637716 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:08.637716 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:08.637716 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:08.637716 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:08.637716 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.637716 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:08.638212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.638212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:08.638212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:08.638212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:08.638212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.638212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:25:08.638212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:08.638212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:08.638212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:08.638212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:08.638212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:08.638212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:08.638795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:08.638795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:08.638795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:08.638795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:08.638795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:08.638795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:08.638795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:08.638795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:08.638795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:08.638795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:08.638795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:08.638795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:08.638795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:08.639413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.639413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:08.639413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.639413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:08.639413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:08.639413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:08.639413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:08.639413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:08.639413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.639413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:08.639413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:08.639413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.639413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:08.639413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:08.639413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.640013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:08.640013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:08.640013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.640013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:25:08.640013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:08.640013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:08.640013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:08.640013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:08.640013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:08.640013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:08.640013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:08.640013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:08.640013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:08.640013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:08.640013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:08.640013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:08.640013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:08.640013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:08.640013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:08.640607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:08.640607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:08.640607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:08.640607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:08.640607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.640607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:08.834142 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:08.834564 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:08.834646 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:08.835557 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:08.835897 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:08.836068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.836717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:08.836717 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:08.836717 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:08.836717 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:08.836717 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:08.836717 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:08.836717 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.836717 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:08.837649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.837649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:08.837649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:08.837649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:08.837649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.837649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:25:08.837649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:08.837649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:08.837649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:08.837649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:08.837649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:08.837649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:08.837932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:08.837932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:08.837932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:08.837932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:08.837932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:08.837932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:08.837932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:08.837932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:08.837932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:08.837932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:08.837932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:08.837932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:08.837932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:08.838171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.838171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:08.838216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.838216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:08.838216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:08.838216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:08.838216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:08.838216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:08.838216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.838216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:08.838216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:08.838216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.838216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:08.838216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:08.838216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.838216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:08.838216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:08.838216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.838519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:25:08.838519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:08.838519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:08.838519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:08.838519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:08.838519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:08.838519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:08.838519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:08.838519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:08.838519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:08.838519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:08.838519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:08.838519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:08.838519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:08.838519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:08.838519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:08.838519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:08.838519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:08.838519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:08.838922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:08.838922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.838922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:08.891330 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:08.892253 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:08.892325 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:08.892450 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:25:08.892876 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:08.893103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.894330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:08.894330 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:08.894330 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:08.894330 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:08.894330 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:08.894330 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:08.894330 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.894330 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:08.895239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.895239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:25:08.895239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:08.895239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:08.895239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:08.895239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:08.895239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:08.895239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:08.895239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:08.895239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:08.895239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:08.895239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:25:08.895239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:08.895239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:25:08.895239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:25:08.895746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:08.895746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:25:08.895746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:08.895746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:25:08.895746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:25:08.895746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:08.895746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:25:08.895746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.895746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:25:08.895959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.895959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:08.895959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:08.895959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:08.895959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.895959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:25:08.895959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:25:08.895959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:25:08.895959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:08.895959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:08.895959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:08.895959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:08.896263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:08.896263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:08.896263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:08.896263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:08.896263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:08.896263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:08.896263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:08.896263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:08.896263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:08.896263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:08.896263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:08.896263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:08.896263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:08.896263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:08.896613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:08.896613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:08.896613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.896613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:08.896613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.896613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:08.896613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:08.896613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:08.896613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:08.896613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:08.896613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.896613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:08.896613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:08.896613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.897014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:08.897014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:08.897014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.897014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:08.897014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:08.897014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.897014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:25:08.897014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:25:08.897014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:25:08.897014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:08.897014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:08.897014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:08.897014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:08.897014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:08.897014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:08.897014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:08.897014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:08.897014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:08.897014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:08.897487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:08.897487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:08.897487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:08.897487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:08.897487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:08.897487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:08.897487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:08.897487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:08.897487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:08.897487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:08.897487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.897487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:08.911716 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:08.911955 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:08.912156 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:08.912361 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:08.912776 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:08.913044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.913666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:08.913666 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:08.913666 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:08.913666 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:08.913666 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:08.913666 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:08.913666 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.913666 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:08.915025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.915025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:25:08.915025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:08.915025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:08.915025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:08.915025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:08.915025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:08.915025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:08.915025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:08.915025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:08.915025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:08.915025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:25:08.915025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:08.915025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:25:08.915025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:25:08.915294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:08.915294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:25:08.915294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:08.915294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:25:08.915294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:25:08.915294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:08.915294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:25:08.915294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.915294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:25:08.916725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.916725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:08.916725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:08.916725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:08.916725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.916725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:25:08.916725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:25:08.916725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:08.916725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:08.916725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:08.916725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:08.917092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:08.917092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:08.917092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:08.917092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:08.917092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:08.917092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:08.917092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:08.917092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:08.917092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:08.917092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:08.917092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:08.917092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:08.917092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:08.917092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:08.917397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:08.917397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:08.917397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.917397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:08.920789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.920789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:08.920789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:08.920789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:08.920789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:08.920789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:08.920789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.920789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:08.920789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:08.920789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.920789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:08.920789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:08.920789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.920789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:08.920789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:08.920789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.920789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:25:08.921170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:25:08.921170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:08.921170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:08.921170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:08.921170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:08.921170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:08.921170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:08.921170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:08.921170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:08.921170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:08.921170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:08.921170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:08.921170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:08.921170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:08.921170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:08.921170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:08.921170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:08.921170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:08.921170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:08.921517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:08.921517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:08.921517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.921517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:08.921517 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:08.921517 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:08.921647 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:08.922700 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:08.922700 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:08.922777 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:08.923111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.923891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:08.923891 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:08.923891 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:08.923891 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:08.923891 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:08.923891 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:08.923891 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.923891 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:08.925182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.925182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:08.925182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:08.925182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:08.925182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.925182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:25:08.925182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:08.925182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:08.925182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:08.925182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:08.925182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:08.925182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:08.925460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:08.925460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:08.925460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:08.925460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:08.925460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:08.925460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:08.925460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:08.925460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:08.925460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:08.925460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:08.925460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:08.925460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:08.925460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:08.925723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.925723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:08.926313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.926313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:08.926313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:08.926313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:08.926313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:08.926313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:08.926313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.926313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:08.926313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:08.926313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.926313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:08.926313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:08.926313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.926313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:08.926313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:08.926313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:08.926653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:25:08.926653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:08.926653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:08.926653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:08.926653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:08.926653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:08.926653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:08.926653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:08.926653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:08.926653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:08.926653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:08.926653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:08.926653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:08.926653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:08.926653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:08.926653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:08.926653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:08.926653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:08.926653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:08.927147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:08.927147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:08.927147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:13.601413 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:13.602161 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:13.602307 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:13.603213 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:13.630481 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:13.631073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.631768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:13.631768 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:13.631768 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:13.631768 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:13.631768 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:13.631768 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:13.631768 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.631768 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:13.632652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.632652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:13.632652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:13.632652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:13.632652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.632652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:25:13.632652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:13.632652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:13.632652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:13.632652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:13.632652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:13.632652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:13.633277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:13.633277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:13.633277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:13.633277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:13.633277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:13.633277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:13.633277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:13.633277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:13.633277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:13.633277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:13.633277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:13.633277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:13.633277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:13.633859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.633859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:13.633859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.633859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:13.633859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:13.633859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:13.633859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:13.633859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:13.633859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.633859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:13.633859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:13.633859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.633859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:13.633859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:13.633859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.635535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:13.635535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:13.635535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.635535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:25:13.635535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:13.635535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:13.635535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:13.635535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:13.635535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:13.635535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:13.635535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:13.635535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:13.635535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:13.635535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:13.635535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:13.635535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:13.635535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:13.635535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:13.635535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:13.636470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:13.636470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:13.636470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:13.636470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:13.636470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.636470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:13.834837 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:13.835451 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:13.835646 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:13.836878 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:13.837079 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:13.837269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.837893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:13.837893 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:13.837893 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:13.837893 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:13.837893 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:13.837893 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:13.837893 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.837893 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:13.839249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.839249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:13.839249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:13.839249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:13.839249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.839249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:25:13.839249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:13.839249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:13.839249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:13.839249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:13.839249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:13.839249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:13.839588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:13.839588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:13.839588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:13.839588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:13.839588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:13.839588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:13.839588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:13.839588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:13.839588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:13.839588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:13.839588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:13.839588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:13.839588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:13.839933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.839933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:13.839933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.839933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:13.839933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:13.839933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:13.839933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:13.839933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:13.839933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.839933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:13.839933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:13.839933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.839933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:13.839933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:13.839933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.840376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:13.840376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:13.840376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.840376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:25:13.840376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:13.840376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:13.840376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:13.840376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:13.840376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:13.840376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:13.840376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:13.840376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:13.840376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:13.840376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:13.840376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:13.840376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:13.840376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:13.840376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:13.840376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:13.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:13.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:13.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:13.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:13.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.840813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:13.892789 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:13.893372 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:13.894152 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:13.894152 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:25:13.894254 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:13.895416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.895416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:13.895416 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:13.895416 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:13.895416 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:13.895416 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:13.895416 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:13.895416 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.895416 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:13.895916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.895916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:25:13.895916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:13.895916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:13.895916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:13.895916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:13.895916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:13.895916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:13.895916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:13.895916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:13.895916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:13.895916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:25:13.895916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:13.895916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:25:13.895916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:25:13.896364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:13.896364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:25:13.896364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:13.896364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:25:13.896364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:25:13.896364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:13.896364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:25:13.896364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.896364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:25:13.896802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.896802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:13.896802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:13.896802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:13.896802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.896802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:25:13.896802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:25:13.896802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:25:13.896802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:13.896802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:13.896802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:13.896802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:13.897187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:13.897187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:13.897187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:13.897187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:13.897187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:13.897187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:13.897187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:13.897187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:13.897187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:13.897187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:13.897187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:13.897187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:13.897187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:13.897187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:13.897591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:13.897591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:13.897591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.897591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:13.898776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.898776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:13.898776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:13.898776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:13.898776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:13.898776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:13.898776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.898776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:13.898776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:13.898776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.898776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:13.898776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:13.898776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.898776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:13.898776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:13.898776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.899254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:25:13.899254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:25:13.899254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:25:13.899254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:13.899254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:13.899254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:13.899254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:13.899254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:13.899254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:13.899254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:13.899254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:13.899254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:13.899254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:13.899254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:13.899254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:13.899254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:13.899254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:13.899254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:13.899729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:13.899729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:13.899729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:13.899729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:13.899729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:13.899729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.899729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:13.912100 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:13.912877 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:13.912996 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:13.913150 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:13.913634 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:13.913920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.914490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:13.914490 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:13.914490 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:13.914490 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:13.914490 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:13.914490 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:13.914490 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.914490 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:13.915467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.915467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:25:13.915467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:13.915467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:13.915467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:13.915467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:13.915467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:13.915467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:13.915467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:13.915467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:13.915467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:13.915467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:25:13.915467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:13.915467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:25:13.915467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:25:13.915963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:13.915963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:25:13.915963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:13.915963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:25:13.915963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:25:13.915963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:13.915963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:25:13.915963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.915963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:25:13.916371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.916371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:13.916371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:13.916371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:13.916371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.916371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:25:13.916371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:25:13.916371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:13.916371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:13.916371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:13.916371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:13.916969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:13.916969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:13.916969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:13.916969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:13.916969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:13.916969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:13.916969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:13.916969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:13.916969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:13.916969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:13.916969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:13.916969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:13.916969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:13.916969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:13.917522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:13.917522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:13.917522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.917522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:13.917522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.917522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:13.917522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:13.917522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:13.917522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:13.917522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:13.917522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.917522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:13.917522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:13.917522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.917971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:13.917971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:13.917971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.917971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:13.917971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:13.917971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.917971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:25:13.917971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:25:13.917971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:13.917971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:13.917971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:13.917971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:13.917971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:13.917971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:13.917971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:13.917971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:13.917971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:13.917971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:13.917971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:13.918954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:13.918954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:13.918954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:13.918954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:13.918954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:13.918954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:13.918954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:13.918954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:13.918954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:13.918954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.918954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:13.921717 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:13.922044 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:13.922106 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:13.922247 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:13.922619 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:13.922883 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:13.923056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.923558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:13.923558 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:13.923558 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:13.923558 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:13.923558 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:13.923558 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:13.923558 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.923558 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:13.924232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.924232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:13.924232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:13.924232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:13.924232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.924232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:25:13.924232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:13.924232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:13.924232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:13.924232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:13.924232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:13.924232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:13.924648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:13.924648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:13.924648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:13.924648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:13.924648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:13.924648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:13.924648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:13.924648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:13.924648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:13.924648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:13.924648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:13.924648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:13.924648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:13.925057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.925057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:13.925057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.925057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:13.925057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:13.925057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:13.925057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:13.925057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:13.925057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.925057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:13.925057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:13.925057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.925057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:13.925057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:13.925057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.925468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:13.925468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:13.925468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:13.925468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:25:13.925468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:13.925468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:13.925468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:13.925468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:13.925468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:13.925468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:13.925468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:13.925468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:13.925468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:13.925468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:13.925468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:13.925468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:13.925468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:13.925468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:13.925468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:13.925944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:13.925944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:13.925944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:13.925944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:13.925944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:13.925944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:18.601951 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:18.602770 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:18.602770 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:18.603516 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:18.635983 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:18.636281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.637119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:18.637119 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:18.637119 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:18.637119 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:18.637119 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:18.637119 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:18.637119 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.637119 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:18.637959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.637959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:18.637959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:18.637959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:18.637959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.637959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:25:18.637959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:18.637959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:18.637959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:18.637959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:18.637959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:18.637959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:18.638507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:18.638507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:18.638507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:18.638507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:18.638507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:18.638507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:18.638507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:18.638507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:18.638507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:18.638507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:18.638507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:18.638507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:18.638507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:18.639052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.639052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:18.639052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.639052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:18.639052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:18.639052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:18.639052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:18.639052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:18.639052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.639052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:18.639052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:18.639052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.639052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:18.639052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:18.639052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.639657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:18.639657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:18.639657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.639657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:25:18.639657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:18.639657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:18.639657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:18.639657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:18.639657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:18.639657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:18.639657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:18.639657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:18.639657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:18.639657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:18.639657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:18.639657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:18.639657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:18.639657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:18.639657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:18.640642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:18.640642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:18.640642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:18.640642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:18.640642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.640642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:18.836338 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:18.836532 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:18.836703 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:18.839360 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:18.840079 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:18.840079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.840928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:18.840928 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:18.840928 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:18.840928 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:18.840928 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:18.840928 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:18.840928 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.840928 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:18.842470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.842470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:18.842470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:18.842470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:18.842470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.842470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:25:18.842470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:18.842470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:18.842470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:18.842470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:18.842470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:18.842470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:18.842947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:18.842947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:18.842947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:18.842947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:18.842947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:18.842947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:18.842947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:18.842947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:18.842947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:18.842947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:18.842947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:18.842947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:18.842947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:18.843342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.843342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:18.844398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.844398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:18.844398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:18.844398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:18.844398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:18.844398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:18.844398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.844398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:18.844398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:18.844398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.844398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:18.844398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:18.844398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.844398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:18.844398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:18.844398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.845027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:25:18.845027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:18.845027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:18.845027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:18.845027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:18.845027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:18.845027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:18.845027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:18.845027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:18.845027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:18.845027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:18.845027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:18.845027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:18.845027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:18.845027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:18.845027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:18.845027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:18.845027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:18.845027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:18.845587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:18.845587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.845587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:18.893363 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:18.893881 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:18.894143 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:18.894356 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:25:18.894978 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:18.895277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.896237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:18.896237 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:18.896237 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:18.896237 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:18.896237 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:18.896237 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:18.896237 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.896237 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:18.897191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.897191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:25:18.897191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:18.897191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:18.897191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:18.897191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:18.897191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:18.897191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:18.897191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:18.897191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:18.897191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:18.897191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:25:18.897191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:18.897191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:25:18.897191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:25:18.897797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:18.897797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:25:18.897797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:18.897797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:25:18.897797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:25:18.897797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:18.897797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:25:18.897797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.897797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:25:18.898956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.898956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:18.898956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:18.898956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:18.898956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.898956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:25:18.898956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:25:18.898956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:25:18.898956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:18.898956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:18.898956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:18.898956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:18.899436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:18.899436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:18.899436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:18.899436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:18.899436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:18.899436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:18.899436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:18.899436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:18.899436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:18.899436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:18.899436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:18.899436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:18.899436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:18.899436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:18.899936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:18.899936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:18.899936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.899936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:18.900955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.900955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:18.900955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:18.900955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:18.900955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:18.900955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:18.900955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.900955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:18.900955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:18.900955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.900955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:18.900955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:18.900955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.900955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:18.900955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:18.900955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.901627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:25:18.901627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:25:18.901627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:25:18.901627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:18.901627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:18.901627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:18.901627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:18.901627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:18.901627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:18.901627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:18.901627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:18.901627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:18.901627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:18.901627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:18.901627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:18.901627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:18.901627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:18.901627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:18.902258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:18.902258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:18.902258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:18.902258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:18.902258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:18.902258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.902258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:18.912936 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:18.913240 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:18.913361 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:18.913497 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:18.914723 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:18.914723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.914841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:18.914841 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:18.914841 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:18.914841 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:18.914841 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:18.914841 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:18.914841 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.914841 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:18.916100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.916100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:25:18.916100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:18.916100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:18.916100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:18.916100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:18.916100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:18.916100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:18.916100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:18.916100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:18.916100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:18.916100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:25:18.916100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:18.916100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:25:18.916100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:25:18.916714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:18.916714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:25:18.916714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:18.916714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:25:18.916714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:25:18.916714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:18.916714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:25:18.916714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.916714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:25:18.917534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.917534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:18.917534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:18.917534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:18.917534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.917534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:25:18.917534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:25:18.917534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:18.917534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:18.917534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:18.917534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:18.918083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:18.918083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:18.918083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:18.918083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:18.918083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:18.918083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:18.918083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:18.918083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:18.918083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:18.918083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:18.918083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:18.918083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:18.918083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:18.918083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:18.918595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:18.918595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:18.918595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.918595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:18.921730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.921730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:18.921730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:18.921730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:18.921730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:18.921730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:18.921730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.921730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:18.921730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:18.921730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.921730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:18.921730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:18.921730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.921730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:18.921730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:18.921730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.921730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:25:18.922552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:25:18.922552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:18.922552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:18.922552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:18.922552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:18.922552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:18.922552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:18.922552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:18.922552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:18.922552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:18.922552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:18.922552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:18.922552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:18.922552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:18.922552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:18.922552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:18.922552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:18.922552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:18.922552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:18.923293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:18.923293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:18.923293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.923293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:18.923293 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:18.923293 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:18.923293 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:18.923293 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:18.923669 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:18.923749 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:18.924063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.924645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:18.924645 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:18.924645 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:18.924645 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:18.924645 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:18.924645 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:18.924645 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.924645 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:18.925492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.925492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:18.925492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:18.925492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:18.925492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.925492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:25:18.925492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:18.925492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:18.925492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:18.925492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:18.925492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:18.925492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:18.926063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:18.926063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:18.926063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:18.926063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:18.926063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:18.926063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:18.926063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:18.926063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:18.926063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:18.926063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:18.926063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:18.926063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:18.926063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:18.926593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.926593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:18.926593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.926593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:18.926593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:18.926593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:18.926593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:18.926593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:18.926593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.926593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:18.926593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:18.926593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.926593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:18.926593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:18.926593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.927135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:18.927135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:18.927135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:18.927135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:25:18.927135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:18.927135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:18.927135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:18.927135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:18.927135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:18.927135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:18.927135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:18.927135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:18.927135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:18.927135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:18.927135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:18.927135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:18.927135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:18.927135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:18.927135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:18.927623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:18.927623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:18.927623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:18.927623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:18.927623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:18.927623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:23.602583 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:23.603401 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:23.603976 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:23.605186 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:23.653709 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:23.653709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.653709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:23.653709 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:23.653709 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:23.653709 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:23.653709 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:23.653709 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:23.653709 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.653709 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:23.654406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.049 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.654406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:23.654406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:23.654406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:23.654406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.654406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:25:23.654406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:23.654406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:23.654406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:23.654406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:23.654406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:23.654406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:23.654719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:23.654719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:23.654719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:23.654719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:23.654719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:23.654719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:23.654719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:23.654719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:23.654719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:23.654719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:23.654719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:23.654719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:23.654719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:23.654974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.654974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:23.655476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.655476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:23.655476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:23.655476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:23.655476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:23.655476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:23.655476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.655476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:23.655476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:23.655476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.655476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:23.655476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:23.655476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.655476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:23.655476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:23.655476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.656451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:25:23.656451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:23.656451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:23.656451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:23.656451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:23.656451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:23.656451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:23.656451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:23.656451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:23.656451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:23.656451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:23.656451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:23.656451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:23.656451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:23.656451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:23.656451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:23.656451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:23.656451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:23.656451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:23.657713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:23.657713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.657713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:23.836711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:23.837496 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:23.837496 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:23.838922 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:23.838922 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:23.838922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.839074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:23.839074 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:23.839074 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:23.839074 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:23.839074 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:23.839074 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:23.839074 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.839074 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:23.840575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.840575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:23.840575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:23.840575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:23.840575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.840575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:25:23.840575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:23.840575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:23.840575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:23.840575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:23.840575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:23.840575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:23.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:23.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:23.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:23.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:23.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:23.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:23.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:23.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:23.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:23.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:23.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:23.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:23.840851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:23.841111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.841111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:23.841364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.841364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:23.841364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:23.841364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:23.841364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:23.841364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:23.841364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.841364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:23.841364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:23.841364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.841364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:23.841364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:23.841364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.841364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:23.841364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:23.841364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.842008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:25:23.842008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:23.842008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:23.842008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:23.842008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:23.842008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:23.842008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:23.842008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:23.842008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:23.842008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:23.842008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:23.842008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:23.842008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:23.842008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:23.842008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:23.842008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:23.842008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:23.842008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:23.842008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:23.842525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:23.842525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.842525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:23.894097 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:23.894529 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:23.894627 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:23.894719 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:25:23.896509 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:23.896509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.896696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:23.896696 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:23.896696 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:23.896696 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:23.896696 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:23.896696 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:23.896696 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.896696 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:23.897751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.897751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:25:23.897751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:23.897751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:23.897751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:23.897751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:23.897751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:23.897751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:23.897751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:23.897751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:23.897751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:23.897751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:25:23.897751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:23.897751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:25:23.897751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:25:23.898248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:23.898248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:25:23.898248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:23.898248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:25:23.898248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:25:23.898248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:23.898248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:25:23.898248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.898248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:25:23.898525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.898525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:23.898525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:23.898525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:23.898525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.898525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:25:23.898525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:25:23.898525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:25:23.898525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:23.898525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:23.898525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:23.898525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:23.898796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:23.898796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:23.898796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:23.898796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:23.898796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:23.898796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:23.898796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:23.898796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:23.898796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:23.898796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:23.898796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:23.898796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:23.898796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:23.898796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:23.899062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:23.899062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:23.899062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.899062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:23.899489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.899489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:23.899489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:23.899489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:23.899489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:23.899489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:23.899489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.899489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:23.899489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:23.899489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.899489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:23.899489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:23.899489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.899489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:23.899489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:23.899489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.899842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:25:23.899842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:25:23.899842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:25:23.899842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:23.899842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:23.899842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:23.899842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:23.899842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:23.899842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:23.899842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:23.899842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:23.899842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:23.899842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:23.899842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:23.899842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:23.899842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:23.899842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:23.899842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:23.900173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:23.900173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:23.900173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:23.900173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:23.900173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:23.900173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.900173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:23.913467 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:23.913755 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:23.913826 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:23.913963 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:23.914491 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:23.915088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.916169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:23.916169 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:23.916169 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:23.916169 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:23.916169 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:23.916169 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:23.916169 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.916169 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:23.916860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.916860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:25:23.916860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:23.916860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:23.916860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:23.916860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:23.916860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:23.916860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:23.916860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:23.916860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:23.916860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:23.916860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:25:23.916860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:23.916860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:25:23.916860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:25:23.917523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:23.917523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:25:23.917523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:23.917523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:25:23.917523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:25:23.917523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:23.917523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:25:23.917523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.917523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:25:23.917925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.917925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:23.917925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:23.917925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:23.917925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.917925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:25:23.917925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:25:23.917925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:23.917925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:23.917925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:23.917925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:23.918400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:23.918400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:23.918400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:23.918400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:23.918400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:23.918400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:23.918400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:23.918400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:23.918400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:23.918400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:23.918400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:23.918400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:23.918400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:23.918400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:23.919199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:23.919199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:23.919199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.919199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:23.919199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.919199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:23.919199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:23.919199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:23.919199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:23.919199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:23.919199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.919199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:23.919199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:23.919199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.920044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:23.920044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:23.920044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.920044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:23.920044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:23.920044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.920044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:25:23.920044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:25:23.920044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:23.920044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:23.920044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:23.920044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:23.920044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:23.920044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:23.920044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:23.920044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:23.920044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:23.920044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:23.920044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:23.920922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:23.920922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:23.920922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:23.920922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:23.920922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:23.920922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:23.920922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:23.920922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:23.920922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:23.920922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.920922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:23.922587 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:23.922859 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:23.922955 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:23.923099 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:23.923757 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:23.924506 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:23.924506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.924804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:23.924804 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:23.924804 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:23.924804 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:23.924804 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:23.924804 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:23.924804 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.924804 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:23.925552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.925552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:23.925552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:23.925552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:23.925552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.925552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:25:23.925552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:23.925552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:23.925552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:23.925552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:23.925552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:23.925552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:23.925826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:23.925826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:23.925826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:23.925826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:23.925826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:23.925826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:23.925826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:23.925826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:23.925826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:23.925826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:23.925826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:23.925826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:23.925826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:23.926083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.926083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:23.926501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.926501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:23.926501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:23.926501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:23.926501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:23.926501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:23.926501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.926501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:23.926501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:23.926501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.926501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:23.926501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:23.926501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.926501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:23.926501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:23.926501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:23.926920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:25:23.926920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:23.926920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:23.926920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:23.926920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:23.926920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:23.926920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:23.926920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:23.926920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:23.926920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:23.926920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:23.926920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:23.926920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:23.926920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:23.926920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:23.926920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:23.926920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:23.926920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:23.926920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:23.927294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:23.927294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:23.927294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:28.603106 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:28.603615 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:28.603615 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:28.604438 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:28.637667 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:28.638098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.639150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:28.639150 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:28.639150 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:28.639150 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:28.639150 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:28.639150 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:28.639150 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.639150 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:28.640381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.640381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:28.640381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:28.640381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:28.640381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.640381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:25:28.640381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:28.640381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:28.640381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:28.640381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:28.640381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:28.640381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:28.640822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:28.640822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:28.640822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:28.640822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:28.640822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:28.640822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:28.640822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:28.640822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:28.640822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:28.640822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:28.640822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:28.640822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:28.640822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:28.641186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.641186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:28.641253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.641253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:28.641253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:28.641253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:28.641253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:28.641253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:28.641253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.641253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:28.641253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:28.641253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.641253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:28.641253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:28.641253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.641253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:28.641253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:28.641253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.641766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:25:28.641766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:28.641766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:28.641766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:28.641766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:28.641766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:28.641766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:28.641766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:28.641766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:28.641766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:28.641766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:28.641766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:28.641766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:28.641766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:28.641766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:28.641766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:28.641766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:28.641766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:28.641766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:28.642281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:28.642281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.642281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:28.837347 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:28.837661 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:28.837813 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:28.838906 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:28.839279 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:28.839547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.840582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:28.840582 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:28.840582 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:28.840582 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:28.840582 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:28.840582 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:28.840582 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.840582 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:28.841656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.841656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:28.841656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:28.841656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:28.841656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.841656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:25:28.841656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:28.841656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:28.841656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:28.841656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:28.841656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:28.841656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:28.842298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:28.842298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:28.842298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:28.842298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:28.842298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:28.842298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:28.842298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:28.842298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:28.842298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:28.842298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:28.842298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:28.842298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:28.842298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:28.842915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.842915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:28.842915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.842915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:28.842915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:28.842915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:28.842915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:28.842915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:28.842915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.842915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:28.842915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:28.842915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.842915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:28.842915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:28.842915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.843665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:28.843665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:28.843665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.843665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:25:28.843665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:28.843665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:28.843665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:28.843665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:28.843665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:28.843665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:28.843665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:28.843665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:28.843665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:28.843665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:28.843665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:28.843665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:28.843665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:28.843665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:28.843665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:28.844586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:28.844586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:28.844586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:28.844586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:28.844586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.844586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:28.894622 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:28.895066 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:28.895190 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:28.895858 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:25:28.896142 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:28.896554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.899801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:28.899801 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:28.899801 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:28.899801 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:28.899801 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:28.899801 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:28.899801 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.899801 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:28.899801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.899801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:25:28.899801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:28.899801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:28.899801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:28.899801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:28.899801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:28.899801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:28.899801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:28.899801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:28.901096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:28.901096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:25:28.901096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:28.901096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:25:28.901096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:25:28.901096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:28.901096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:25:28.901096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:28.901096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:25:28.901096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:25:28.901096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:28.901096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:25:28.901096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.901096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:25:28.901096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.901905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:28.901905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:28.901905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:28.901905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.901905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:25:28.901905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:25:28.901905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:25:28.901905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:28.901905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:28.901905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:28.901905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:28.901905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:28.901905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:28.902518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:28.902518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:28.902518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:28.902518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:28.902518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:28.902518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:28.902518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:28.902518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:28.902518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:28.902518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:28.902518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:28.902518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:28.902518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:28.902518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:28.903193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.903193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:28.903193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.903193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:28.903193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:28.903193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:28.903193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:28.903193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:28.903193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.903193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:28.903193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:28.903193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.903193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:28.903193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:28.903193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.903918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:28.903918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:28.903918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.903918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:25:28.903918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:25:28.903918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:25:28.903918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:28.903918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:28.903918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:28.903918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:28.903918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:28.903918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:28.903918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:28.903918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:28.903918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:28.903918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:28.903918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:28.903918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:28.903918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:28.904828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:28.904828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:28.904828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:28.904828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:28.904828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:28.904828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:28.904828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:28.904828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.904828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:28.914039 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:28.914387 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:28.914506 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:28.914612 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:28.915817 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:28.915817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.916006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:28.916006 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:28.916006 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:28.916006 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:28.916006 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:28.916006 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:28.916006 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.916006 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:28.916694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.916694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:25:28.916694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:28.916694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:28.916694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:28.916694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:28.916694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:28.916694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:28.916694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:28.916694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:28.916694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:28.916694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:25:28.916694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:28.916694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:25:28.916694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:25:28.917453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:28.917453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:25:28.917453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:28.917453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:25:28.917453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:25:28.917453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:28.917453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:25:28.917453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.917453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:25:28.918007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.918007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:28.918007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:28.918007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:28.918007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.918007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:25:28.918007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:25:28.918007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:28.918007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:28.918007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:28.918007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:28.918743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:28.918743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:28.918743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:28.918743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:28.918743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:28.918743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:28.918743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:28.918743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:28.918743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:28.918743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:28.918743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:28.918743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:28.918743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:28.918743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:28.919553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:28.919553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:28.919553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.919553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:28.919553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.919553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:28.919553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:28.919553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:28.919553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:28.919553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:28.919553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.919553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:28.919553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:28.919553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.920525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:28.920525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:28.920525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.920525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:28.920525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:28.920525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.920525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:25:28.920525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:25:28.920525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:28.920525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:28.920525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:28.920525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:28.920525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:28.920525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:28.920525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:28.920525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:28.920525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:28.920525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:28.920525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:28.921933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:28.921933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:28.921933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:28.921933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:28.921933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:28.921933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:28.921933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:28.921933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:28.921933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:28.921933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.921933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:28.923186 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:28.923484 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:28.923615 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:28.923816 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:28.924128 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:28.924412 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:28.924620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.925190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:28.925190 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:28.925190 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:28.925190 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:28.925190 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:28.925190 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:28.925190 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.925190 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:28.926150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.926150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:28.926150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:28.926150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:28.926150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.926150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:25:28.926150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:28.926150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:28.926150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:28.926150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:28.926150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:28.926150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:28.926527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:28.926527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:28.926527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:28.926527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:28.926527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:28.926527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:28.926527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:28.926527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:28.926527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:28.926527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:28.926527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:28.926527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:28.926527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:28.926867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.926867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:28.927330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.927330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:28.927330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:28.927330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:28.927330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:28.927330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:28.927330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.927330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:28.927330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:28.927330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.927330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:28.927330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:28.927330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.927330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:28.927330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:28.927330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:28.927786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:25:28.927786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:28.927786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:28.927786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:28.927786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:28.927786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:28.927786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:28.927786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:28.927786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:28.927786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:28.927786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:28.927786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:28.927786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:28.927786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:28.927786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:28.927786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:28.927786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:28.927786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:28.927786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:28.928287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:28.928287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:28.928287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:33.603663 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:33.604474 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:33.604474 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:33.605567 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:33.637813 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:33.637813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.640265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:33.640265 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:33.640265 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:33.640265 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:33.640265 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:33.640265 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:33.640265 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.640265 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:33.640265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.640265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:33.640265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:33.640265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:33.640265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.640265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:25:33.640265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:33.640747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:33.640747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:33.640747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:33.640747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:33.640747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:33.640747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:33.640747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:33.640747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:33.640747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:33.640747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:33.640747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:33.640747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:33.640747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:33.640747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:33.641133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:33.641133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:33.641133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:33.641133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:33.641133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.641133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:33.641133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.641133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:33.641133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:33.641133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:33.641133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:33.641133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:33.641133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.641133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:33.641505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:33.641505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.641505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:33.641505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:33.641505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.641505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:33.641505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:33.641505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.641505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:25:33.641505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:33.641505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:33.641505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:33.641505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:33.641505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:33.641505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:33.641505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:33.641505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:33.641505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:33.641505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:33.641505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:33.641505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:33.642072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:33.642072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:33.642072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:33.642072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:33.642072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:33.642072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:33.642072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:33.642072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.642072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:33.837987 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:33.838941 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:33.838941 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:33.839915 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:33.840159 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:33.840768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.841207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:33.841207 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:33.841207 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:33.841207 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:33.841207 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:33.841207 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:33.841207 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.841207 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:33.842453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.842453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:33.842453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:33.842453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:33.842453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.842453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:25:33.842453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:33.842453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:33.842453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:33.842453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:33.842453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:33.842453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:33.843082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:33.843082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:33.843082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:33.843082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:33.843082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:33.843082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:33.843082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:33.843082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:33.843082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:33.843082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:33.843082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:33.843082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:33.843082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:33.843640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.843640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:33.843640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.843640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:33.843640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:33.843640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:33.843640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:33.843640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:33.843640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.843640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:33.843640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:33.843640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.843640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:33.843640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:33.843640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.844270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:33.844270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:33.844270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.844270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:25:33.844270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:33.844270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:33.844270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:33.844270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:33.844270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:33.844270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:33.844270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:33.844270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:33.844270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:33.844270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:33.844270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:33.844270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:33.844270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:33.844270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:33.844270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:33.845021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:33.845021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:33.845021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:33.845021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:33.845021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.845021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:33.895143 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:33.895577 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:33.895700 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:33.895859 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:25:33.896435 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:33.896657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.897295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:33.897295 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:33.897295 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:33.897295 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:33.897295 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:33.897295 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:33.897295 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.897295 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:33.898160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.898160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:25:33.898160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:33.898160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:33.898160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:33.898160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:33.898160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:33.898160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:33.898160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:33.898160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:33.898160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:33.898160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:25:33.898160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:33.898160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:25:33.898160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:25:33.898963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:33.898963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:25:33.898963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:33.898963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:25:33.898963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:25:33.898963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:33.898963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:25:33.898963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.898963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:25:33.899423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.899423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:33.899423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:33.899423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:33.899423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.899423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:25:33.899423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:25:33.899423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:25:33.899423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:33.899423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:33.899423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:33.899423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:33.899998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:33.899998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:33.899998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:33.899998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:33.899998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:33.899998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:33.899998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:33.899998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:33.899998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:33.899998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:33.899998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:33.899998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:33.899998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:33.899998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:33.900627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:33.900627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:33.900627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.900627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:33.900627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.900627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:33.900627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:33.900627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:33.900627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:33.900627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:33.900627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.900627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:33.900627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:33.900627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.901310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:33.901310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:33.901310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.901310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:33.901310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:33.901310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.901310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:25:33.901310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:25:33.901310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:25:33.901310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:33.901310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:33.901310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:33.901310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:33.901310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:33.901310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:33.901310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:33.901310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:33.901310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:33.901310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:33.902143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:33.902143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:33.902143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:33.902143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:33.902143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:33.902143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:33.902143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:33.902143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:33.902143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:33.902143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:33.902143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.902143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:33.914583 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:33.915017 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:33.915140 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:33.915321 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:33.915949 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:33.916338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.917146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:33.917146 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:33.917146 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:33.917146 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:33.917146 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:33.917146 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:33.917146 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.917146 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:25:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:25:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:25:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:25:33.918620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:33.918620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:25:33.918620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:33.918620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:25:33.918620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:25:33.918620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:33.918620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:25:33.918620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.918620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:25:33.919310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.919310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:33.919310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:33.919310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:33.919310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.919310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:25:33.919310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:25:33.919310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:33.919310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:33.919310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:33.919310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:33.920017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:33.920017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:33.920017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:33.920017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:33.920017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:33.920017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:33.920017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:33.920017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:33.920017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:33.920017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:33.920017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:33.920017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:33.920017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:33.920017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:33.920647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:33.920647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:33.920647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.920647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:33.920647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.920647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:33.920647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:33.920647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:33.920647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:33.920647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:33.920647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.920647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:33.920647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:33.920647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.921290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:33.921290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:33.921290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.921290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:33.921290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:33.921290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.921290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:25:33.921290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:25:33.921290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:33.921290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:33.921290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:33.921290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:33.921290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:33.921290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:33.921290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:33.921290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:33.921290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:33.921290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:33.921290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:33.922137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:33.922137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:33.922137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:33.922137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:33.922137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:33.922137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:33.922137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:33.922137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:33.922137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:33.922137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.922137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:33.923574 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:33.924121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:33.924345 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:33.924747 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:33.925484 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:33.925722 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:33.925874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.926441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:33.926441 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:33.926441 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:33.926441 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:33.926441 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:33.926441 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:33.926441 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.926441 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:33.927802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.927802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:33.927802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:33.927802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:33.927802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.927802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:25:33.927802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:33.927802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:33.927802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:33.927802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:33.927802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:33.927802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:33.928154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:33.928154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:33.928154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:33.928154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:33.928154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:33.928154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:33.928154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:33.928154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:33.928154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:33.928154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:33.928154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:33.928154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:33.928154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:33.928481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.928481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:33.929288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.929288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:33.929288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:33.929288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:33.929288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:33.929288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:33.929288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.929288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:33.929288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:33.929288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.929288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:33.929288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:33.929288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.929288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:33.929288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:33.929288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:33.929799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:25:33.929799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:33.929799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:33.929799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:33.929799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:33.929799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:33.929799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:33.929799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:33.929799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:33.929799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:33.929799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:33.929799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:33.929799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:33.929799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:33.929799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:33.929799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:33.929799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:33.929799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:33.929799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:33.930347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:33.930347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:33.930347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:38.604214 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:38.604900 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:38.605057 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:38.606496 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:38.640344 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:38.640588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.641204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:38.641204 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:38.641204 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:38.641204 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:38.641204 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:38.641204 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:38.641204 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.641204 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:38.642230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.642230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:38.642230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:38.642230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:38.642230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.642230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:25:38.642230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:38.642230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:38.642230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:38.642230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:38.642230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:38.642230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:38.642619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:38.642619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:38.642619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:38.642619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:38.642619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:38.642619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:38.642619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:38.642619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:38.642619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:38.642619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:38.642619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:38.642619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:38.642619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:38.643039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.643039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:38.643320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.643320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:38.643320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:38.643320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:38.643320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:38.643320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:38.643320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.643320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:38.643320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:38.643320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.643320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:38.643320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:38.643320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.643320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:38.643320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:38.643320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.643822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:25:38.643822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:38.643822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:38.643822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:38.643822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:38.643822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:38.643822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:38.643822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:38.643822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:38.643822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:38.643822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:38.643822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:38.643822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:38.643822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:38.643822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:38.643822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:38.643822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:38.643822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:38.643822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:38.644358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:38.644358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.644358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:38.838768 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:38.839212 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:38.839327 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:38.840239 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:38.840525 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:38.840653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.842051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:38.842051 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:38.842051 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:38.842051 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:38.842051 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:38.842051 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:38.842051 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.842051 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:38.843322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.843322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:38.843322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:38.843322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:38.843322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.843322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:25:38.843322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:38.843322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:38.843322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:38.843322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:38.843322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:38.843322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:38.843669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:38.843669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:38.843669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:38.843669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:38.843669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:38.843669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:38.843669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:38.843669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:38.843669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:38.843669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:38.843669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:38.843669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:38.843669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:38.844008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.844008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:38.844269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.844269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:38.844269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:38.844269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:38.844269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:38.844269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:38.844269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.844269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:38.844269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:38.844269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.844269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:38.844269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:38.844269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.844269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:38.844269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:38.844269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.844759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:25:38.844759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:38.844759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:38.844759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:38.844759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:38.844759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:38.844759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:38.844759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:38.844759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:38.844759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:38.844759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:38.844759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:38.844759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:38.844759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:38.844759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:38.844759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:38.844759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:38.844759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:38.844759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:38.845225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:38.845225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.845225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:38.895668 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:38.896221 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:38.896402 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:38.896532 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:25:38.897171 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:38.897396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.898283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:38.898283 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:38.898283 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:38.898283 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:38.898283 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:38.898283 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:38.898283 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.898283 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:38.899635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.899635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:25:38.899635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:38.899635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:38.899635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:38.899635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:38.899635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:38.899635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:38.899635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:38.899635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:38.899635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:38.899635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:25:38.899635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:38.899635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:25:38.899635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:25:38.900483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:38.900483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:25:38.900483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:38.900483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:25:38.900483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:25:38.900483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:38.900483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:25:38.900483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.900483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:25:38.901207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.901207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:38.901207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:38.901207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:38.901207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.901207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:25:38.901207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:25:38.901207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:25:38.901207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:38.901207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:38.901207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:38.901207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:38.901852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:38.901852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:38.901852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:38.901852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:38.901852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:38.901852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:38.901852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:38.901852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:38.901852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:38.901852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:38.901852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:38.901852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:38.901852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:38.901852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:38.902493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:38.902493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:38.902493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.902493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:38.902987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.902987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:38.902987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:38.902987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:38.902987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:38.902987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:38.902987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.902987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:38.902987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:38.902987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.902987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:38.902987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:38.902987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.902987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:38.902987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:38.902987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.903748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:25:38.903748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:25:38.903748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:25:38.903748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:38.903748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:38.903748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:38.903748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:38.903748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:38.903748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:38.903748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:38.903748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:38.903748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:38.903748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:38.903748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:38.903748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:38.903748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:38.903748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:38.903748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:38.904596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:38.904596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:38.904596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:38.904596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:38.904596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:38.904596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.904596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:38.914975 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:38.915387 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:38.915643 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:38.915952 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:38.916632 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:38.917817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.917899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:38.917899 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:38.917899 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:38.917899 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:38.917899 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:38.917899 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:38.917899 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.917899 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:38.919628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.919628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:25:38.919628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:38.919628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:38.919628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:38.919628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:38.919628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:38.919628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:38.919628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:38.919628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:38.919628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:38.919628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:25:38.919628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:38.919628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:25:38.919628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:25:38.920382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:38.920382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:25:38.920382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:38.920382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:25:38.920382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:25:38.920382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:38.920382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:25:38.920382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.920382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:25:38.921217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.921217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:38.921217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:38.921217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:38.921217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.921217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:25:38.921217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:25:38.921217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:38.921217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:38.921217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:38.921217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:38.921581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:38.921581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:38.921581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:38.921581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:38.921581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:38.921581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:38.921581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:38.921581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:38.921581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:38.921581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:38.921581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:38.921581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:38.921581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:38.921581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:38.921982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:38.921982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:38.921982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.921982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:38.922761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.922761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:38.922761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:38.922761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:38.922761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:38.922761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:38.922761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.922761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:38.922761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:38.922761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.922761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:38.922761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:38.922761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.922761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:38.922761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:38.922761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.922761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:25:38.923627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:25:38.923627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:38.923627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:38.923627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:38.923627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:38.923627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:38.923627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:38.923627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:38.923627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:38.923627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:38.923627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:38.923627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:38.923627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:38.923627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:38.923627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:38.923627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:38.923627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:38.923627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:38.923627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:38.924562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:38.924562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:38.924562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.924562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:38.924562 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:38.924562 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:38.924562 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:38.924562 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:38.925068 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:38.925433 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:38.925724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.926406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:38.926406 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:38.926406 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:38.926406 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:38.926406 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:38.926406 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:38.926406 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.926406 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:38.927310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.927310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:38.927310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:38.927310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:38.927310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.927310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:25:38.927310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:38.927310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:38.927310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:38.927310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:38.927310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:38.927310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:38.927663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:38.927663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:38.927663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:38.927663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:38.927663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:38.927663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:38.927663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:38.927663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:38.927663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:38.927663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:38.927663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:38.927663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:38.927663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:38.928003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.928003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:38.928457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.928457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:38.928457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:38.928457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:38.928457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:38.928457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:38.928457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.928457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:38.928457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:38.928457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.928457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:38.928457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:38.928457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.928457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:38.928457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:38.928457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:38.929013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:25:38.929013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:38.929013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:38.929013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:38.929013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:38.929013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:38.929013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:38.929013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:38.929013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:38.929013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:38.929013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:38.929013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:38.929013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:38.929013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:38.929013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:38.929013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:38.929013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:38.929013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:38.929013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:38.929497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:38.929497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:38.929497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:43.604842 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:43.605485 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:43.605626 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:43.606500 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:43.643299 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:43.643558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.644234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:43.644234 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:43.644234 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:43.644234 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:43.644234 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:43.644234 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:43.644234 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.644234 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:43.645358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.645358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:43.645358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:43.645358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:43.645358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.645358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:25:43.645358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:43.645358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:43.645358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:43.645358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:43.645358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:43.645358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:43.645748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:43.645748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:43.645748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:43.645748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:43.645748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:43.645748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:43.645748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:43.645748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:43.645748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:43.645748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:43.645748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:43.645748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:43.645748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:43.646039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.646039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:43.646436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.646436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:43.646436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:43.646436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:43.646436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:43.646436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:43.646436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.646436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:43.646436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:43.646436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.646436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:43.646436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:43.646436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.646436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:43.646436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:43.646436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.646873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:25:43.646873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:43.646873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:43.646873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:43.646873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:43.646873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:43.646873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:43.646873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:43.646873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:43.646873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:43.646873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:43.646873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:43.646873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:43.646873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:43.646873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:43.646873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:43.646873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:43.646873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:43.646873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:43.647302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:43.647302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.647302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:43.839286 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:43.839878 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:43.839993 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:43.840730 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:43.840998 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:43.841178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.842097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:43.842097 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:43.842097 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:43.842097 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:43.842097 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:43.842097 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:43.842097 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.842097 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:43.843185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.843185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:43.843185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:43.843185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:43.843185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.843185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:25:43.843185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:43.843185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:43.843185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:43.843185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:43.843185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:43.843185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:43.843560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:43.843560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:43.843560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:43.843560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:43.843560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:43.843560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:43.843560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:43.843560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:43.843560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:43.843560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:43.843560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:43.843560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:43.843560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:43.843964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.843964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:43.844225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.844225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:43.844225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:43.844225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:43.844225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:43.844225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:43.844225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.844225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:43.844225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:43.844225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.844225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:43.844225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:43.844225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.844225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:43.844225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:43.844225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.844791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:25:43.844791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:43.844791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:43.844791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:43.844791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:43.844791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:43.844791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:43.844791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:43.844791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:43.844791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:43.844791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:43.844791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:43.844791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:43.844791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:43.844791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:43.844791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:43.844791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:43.844791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:43.844791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:43.845268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:43.845268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.845268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:43.896363 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:43.896658 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:43.896884 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:43.897064 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:25:43.897723 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:43.897962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.898914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:43.898914 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:43.898914 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:43.898914 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:43.898914 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:43.898914 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:43.898914 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.898914 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:43.900310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.900310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:25:43.900310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:43.900310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:43.900310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:43.900310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:43.900310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:43.900310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:43.900310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:43.900310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:43.900310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:43.900310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:25:43.900310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:43.900310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:25:43.900310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:25:43.901380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:43.901380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:25:43.901380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:43.901380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:25:43.901380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:25:43.901380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:43.901380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:25:43.901380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.901380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:25:43.902429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.902429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:43.902429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:43.902429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:43.902429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.902429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:25:43.902429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:25:43.902429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:25:43.902429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:43.902429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:43.902429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:43.902429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:43.903082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:43.903082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:43.903082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:43.903082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:43.903082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:43.903082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:43.903082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:43.903082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:43.903082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:43.903082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:43.903082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:43.903082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:43.903082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:43.903082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:43.904048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:43.904048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:43.904048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.904048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:43.904048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.904048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:43.904048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:43.904048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:43.904048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:43.904048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:43.904048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.904048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:43.904048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:43.904048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.904854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:43.904854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:43.904854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.904854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:43.904854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:43.904854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.904854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:25:43.904854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:25:43.904854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:25:43.904854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:43.904854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:43.904854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:43.904854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:43.904854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:43.904854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:43.904854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:43.904854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:43.904854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:43.904854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:43.905958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:43.905958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:43.905958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:43.905958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:43.905958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:43.905958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:43.905958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:43.905958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:43.905958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:43.905958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:43.905958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.905958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:43.915520 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:43.915960 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:43.916131 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:43.916338 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:43.916775 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:43.917031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.917823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:43.917823 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:43.917823 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:43.917823 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:43.917823 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:43.917823 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:43.917823 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.917823 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:43.919018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.919018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:25:43.919018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:43.919018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:43.919018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:43.919018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:43.919018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:43.919018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:43.919018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:43.919018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:43.919018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:43.919018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:25:43.919018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:43.919018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:25:43.919018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:25:43.920023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:43.920023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:25:43.920023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:43.920023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:25:43.920023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:25:43.920023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:43.920023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:25:43.920023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.920023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:25:43.920961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.920961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:43.920961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:43.920961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:43.920961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.920961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:25:43.920961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:25:43.920961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:43.920961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:43.920961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:43.920961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:43.921567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:43.921567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:43.921567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:43.921567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:43.921567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:43.921567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:43.921567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:43.921567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:43.921567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:43.921567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:43.921567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:43.921567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:43.921567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:43.921567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:43.922547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:43.922547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:43.922547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.922547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:43.923206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.923206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:43.923206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:43.923206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:43.923206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:43.923206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:43.923206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.923206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:43.923206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:43.923206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.923206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:43.923206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:43.923206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.923206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:43.923206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:43.923206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.923206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:25:43.923859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:25:43.923859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:43.923859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:43.923859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:43.923859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:43.923859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:43.923859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:43.923859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:43.923859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:43.923859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:43.923859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:43.923859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:43.923859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:43.923859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:43.923859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:43.923859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:43.923859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:43.923859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:43.923859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:43.924336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:43.924336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:43.924336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.924336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:43.924336 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:43.925555 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:43.925555 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:43.925555 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:43.925817 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:43.926019 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:43.926152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.928899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:43.928899 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:43.928899 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:43.928899 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:43.928899 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:43.928899 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:43.928899 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.928899 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:43.928899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.928899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:43.928899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:43.928899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:43.928899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.928899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:25:43.928899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:43.929350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:43.929350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:43.929350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:43.929350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:43.929350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:43.929350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:43.929350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:43.929350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:43.929350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:43.929350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:43.929350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:43.929350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:43.929350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:43.929350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:43.929824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:43.929824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:43.929824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:43.929824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:43.929824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.929824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:43.929824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.929824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:43.929824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:43.929824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:43.929824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:43.929824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:43.929824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.929824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:43.930262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:43.930262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.930262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:43.930262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:43.930262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.930262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:43.930262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:43.930262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:43.930262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:25:43.930262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:43.930262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:43.930262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:43.930262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:43.930262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:43.930262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:43.930262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:43.930262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:43.930262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:43.930262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:43.930262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:43.930854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:43.930854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:43.930854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:43.930854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:43.930854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:43.930854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:43.930854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:43.930854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:43.930854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:43.930854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:48.606057 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:48.607061 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:48.607598 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:48.607827 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:48.637892 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:48.638181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.638803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:48.638803 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:48.638803 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:48.638803 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:48.638803 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:48.638803 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:48.638803 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.638803 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:48.639808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.639808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:48.639808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:48.639808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:48.639808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.639808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:25:48.639808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:48.639808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:48.639808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:48.639808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:48.639808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:48.639808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:48.640351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:48.640351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:48.640351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:48.640351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:48.640351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:48.640351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:48.640351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:48.640351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:48.640351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:48.640351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:48.640351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:48.640351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:48.640351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:48.640894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.640894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:48.640894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.640894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:48.640894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:48.640894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:48.640894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:48.640894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:48.640894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.640894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:48.640894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:48.640894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.640894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:48.640894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:48.640894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.641482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:48.641482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:48.641482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.641482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:25:48.641482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:48.641482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:48.641482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:48.641482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:48.641482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:48.641482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:48.641482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:48.641482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:48.641482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:48.641482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:48.641482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:48.641482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:48.641482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:48.641482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:48.641482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:48.642251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:48.642251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:48.642251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:48.642251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:48.642251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.642251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:48.840612 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:48.842415 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:48.842415 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:48.842710 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:48.843071 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:48.843308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.844321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:48.844321 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:48.844321 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:48.844321 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:48.844321 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:48.844321 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:48.844321 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.844321 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:48.845947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.845947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:48.845947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:48.845947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:48.845947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.845947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:25:48.845947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:48.845947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:48.845947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:48.845947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:48.845947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:48.845947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:48.846799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:48.846799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:48.846799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:48.846799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:48.846799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:48.846799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:48.846799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:48.846799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:48.846799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:48.846799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:48.846799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:48.846799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:48.846799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:48.847441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.847441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:48.847441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.847441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:48.847441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:48.847441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:48.847441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:48.847441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:48.847441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.847441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:48.847441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:48.847441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.847441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:48.847441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:48.847441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.848153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:48.848153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:48.848153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.848153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:25:48.848153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:48.848153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:48.848153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:48.848153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:48.848153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:48.848153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:48.848153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:48.848153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:48.848153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:48.848153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:48.848153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:48.848153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:48.848153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:48.848153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:48.848153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:48.848972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:48.848972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:48.848972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:48.848972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:48.848972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.848972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:48.896851 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:48.897305 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:48.897465 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:48.897631 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:25:48.898282 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:48.898589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.899391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:48.899391 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:48.899391 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:48.899391 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:48.899391 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:48.899391 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:48.899391 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.899391 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:48.900197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.900197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:25:48.900197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:48.900197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:48.900197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:48.900197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:48.900197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:48.900197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:48.900197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:48.900197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:48.900197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:48.900197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:25:48.900197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:48.900197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:25:48.900197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:25:48.900963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:48.900963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:25:48.900963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:48.900963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:25:48.900963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:25:48.900963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:48.900963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:25:48.900963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.900963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:25:48.901402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.901402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:48.901402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:48.901402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:48.901402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.901402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:25:48.901402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:25:48.901402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:25:48.901402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:48.901402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:48.901402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:48.901402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:48.902025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:48.902025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:48.902025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:48.902025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:48.902025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:48.902025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:48.902025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:48.902025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:48.902025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:48.902025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:48.902025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:48.902025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:48.902025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:48.902025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:48.902614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:48.902614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:48.902614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.902614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:48.902614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.902614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:48.902614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:48.902614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:48.902614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:48.902614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:48.902614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.902614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:48.902614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:48.902614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.903264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:48.903264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:48.903264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.903264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:48.903264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:48.903264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.903264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:25:48.903264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:25:48.903264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:25:48.903264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:48.903264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:48.903264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:48.903264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:48.903264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:48.903264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:48.903264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:48.903264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:48.903264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:48.903264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:48.904103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:48.904103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:48.904103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:48.904103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:48.904103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:48.904103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:48.904103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:48.904103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:48.904103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:48.904103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:48.904103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.904103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:48.915931 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:48.916376 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:48.916578 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:48.916888 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:48.917387 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:48.917763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.918539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:48.918539 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:48.918539 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:48.918539 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:48.918539 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:48.918539 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:48.918539 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.918539 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:48.919822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.919822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:25:48.919822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:48.919822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:48.919822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:48.919822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:48.919822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:48.919822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:48.919822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:48.919822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:48.919822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:48.919822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:25:48.919822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:48.919822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:25:48.919822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:25:48.920488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:48.920488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:25:48.920488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:48.920488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:25:48.920488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:25:48.920488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:48.920488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:25:48.920488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.920488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:25:48.921281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.921281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:48.921281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:48.921281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:48.921281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.921281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:25:48.921281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:25:48.921281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:48.921281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:48.921281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:48.921281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:48.921831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:48.921831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:48.921831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:48.921831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:48.921831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:48.921831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:48.921831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:48.921831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:48.921831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:48.921831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:48.921831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:48.921831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:48.921831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:48.921831 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:48.922465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:48.922465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:48.922465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.922465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:48.923270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.923270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:48.923270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:48.923270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:48.923270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:48.923270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:48.923270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.923270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:48.923270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:48.923270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.923270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:48.923270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:48.923270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.923270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:48.923270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:48.923270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.923270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:25:48.924148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:25:48.924148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:48.924148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:48.924148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:48.924148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:48.924148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:48.924148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:48.924148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:48.924148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:48.924148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:48.924148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:48.924148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:48.924148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:48.924148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:48.924148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:48.924148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:48.924148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:48.924148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:48.924148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:48.925019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:48.925019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:48.925019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.925019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:48.925207 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:48.925369 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:48.925493 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:48.925575 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:48.926066 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:48.926402 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:48.926610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.927339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:48.927339 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:48.927339 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:48.927339 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:48.927339 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:48.927339 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:48.927339 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.927339 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:48.928216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.928216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:48.928216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:48.928216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:48.928216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.928216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:25:48.928216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:48.928216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:48.928216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:48.928216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:48.928216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:48.928216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:48.928872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:48.928872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:48.928872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:48.928872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:48.928872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:48.928872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:48.928872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:48.928872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:48.928872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:48.928872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:48.928872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:48.928872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:48.928872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:48.929500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.929500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:48.929500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.929500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:48.929500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:48.929500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:48.929500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:48.929500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:48.929500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.929500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:48.929500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:48.929500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.929500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:48.929500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:48.929500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.930414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:48.930414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:48.930414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:48.930414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:25:48.930414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:48.930414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:48.930414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:48.930414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:48.930414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:48.930414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:48.930414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:48.930414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:48.930414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:48.930414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:48.930414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:48.930414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:48.930414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:48.930414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:48.930414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:48.931315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:48.931315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:48.931315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:48.931315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:48.931315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:48.931315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:53.607520 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:53.608127 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:53.608127 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:53.608966 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:53.644317 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:53.644645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.645359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:53.645359 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:53.645359 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:53.645359 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:53.645359 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:53.645359 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:53.645359 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.645359 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:53.646160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.646160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:53.646160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:53.646160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:53.646160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.646160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:25:53.646160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:53.646160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:53.646160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:53.646160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:53.646160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:53.646160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:53.646541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:53.646541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:53.646541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:53.646541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:53.646541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:53.646541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:53.646541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:53.646541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:53.646541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:53.646541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:53.646541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:53.646541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:53.646541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:53.646973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.646973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:53.646973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.646973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:53.646973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:53.646973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:53.646973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:53.646973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:53.646973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.646973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:53.646973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:53.646973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.646973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:53.646973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:53.646973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.647440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:53.647440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:53.647440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.647440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:25:53.647440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:53.647440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:53.647440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:53.647440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:53.647440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:53.647440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:53.647440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:53.647440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:53.647440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:53.647440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:53.647440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:53.647440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:53.647440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:53.647440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:53.647440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:53.647996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:53.647996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:53.647996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:53.647996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:53.647996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.647996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:53.841522 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:53.842072 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:53.842247 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:53.843325 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:53.843792 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:53.844054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.845004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:53.845004 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:53.845004 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:53.845004 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:53.845004 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:53.845004 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:53.845004 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.845004 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:53.846491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.846491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:53.846491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:53.846491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:53.846491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.846491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:25:53.846491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:53.846491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:53.846491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:53.846491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:53.846491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:53.846491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:53.847262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:53.847262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:53.847262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:53.847262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:53.847262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:53.847262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:53.847262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:53.847262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:53.847262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:53.847262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:53.847262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:53.847262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:53.847262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:53.848394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.848394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:53.848394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.848394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:53.848394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:53.848394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:53.848394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:53.848394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:53.848394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.848394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:53.848394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:53.848394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.848394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:53.848394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:53.848394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.848892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:53.848892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:53.848892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.848892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:25:53.848892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:53.848892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:53.848892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:53.848892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:53.848892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:53.848892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:53.848892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:53.848892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:53.848892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:53.848892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:53.848892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:53.848892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:53.848892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:53.848892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:53.848892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:53.849395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:53.849395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:53.849395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:53.849395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:53.849395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.849395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:53.897482 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:53.898076 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:53.898291 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:53.898470 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:25:53.899183 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:53.899527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.900366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:53.900366 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:53.900366 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:53.900366 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:53.900366 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:53.900366 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:53.900366 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.900366 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:53.901642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.901642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:25:53.901642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:53.901642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:53.901642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:53.901642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:53.901642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:53.901642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:53.901642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:53.901642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:53.901642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:53.901642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:25:53.901642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:53.901642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:25:53.901642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:25:53.902462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:53.902462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:25:53.902462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:53.902462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:25:53.902462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:25:53.902462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:53.902462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:25:53.902462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.902462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:25:53.903088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.903088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:53.903088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:53.903088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:53.903088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.903088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:25:53.903088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:25:53.903088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:25:53.903088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:53.903088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:53.903088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:53.903088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:53.903824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:53.903824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:53.903824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:53.903824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:53.903824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:53.903824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:53.903824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:53.903824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:53.903824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:53.903824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:53.903824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:53.903824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:53.903824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:53.903824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:53.904423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:53.904423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:53.904423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.904423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:53.904632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.904632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:53.904632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:53.904632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:53.904632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:53.904632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:53.904632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.904632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:53.904632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:53.904632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.904632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:53.904632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:53.904632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.904632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:53.904632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:53.904632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.905597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:25:53.905597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:25:53.905597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:25:53.905597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:53.905597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:53.905597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:53.905597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:53.905597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:53.905597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:53.905597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:53.905597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:53.905597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:53.905597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:53.905597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:53.905597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:53.905597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:53.905597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:53.905597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:53.906451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:53.906451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:53.906451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:53.906451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:53.906451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:53.906451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.906451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:53.916586 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:53.917033 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:53.917208 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:53.917407 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:53.917951 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:53.918298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.919077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:53.919077 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:53.919077 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:53.919077 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:53.919077 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:53.919077 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:53.919077 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.919077 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:53.920277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.920277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:25:53.920277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:53.920277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:53.920277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:53.920277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:53.920277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:53.920277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:53.920277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:53.920277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:53.920277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:53.920277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:25:53.920277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:53.920277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:25:53.920277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:25:53.921011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:53.921011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:25:53.921011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:53.921011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:25:53.921011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:25:53.921011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:53.921011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:25:53.921011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.921011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:25:53.921705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.921705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:53.921705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:53.921705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:53.921705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.921705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:25:53.921705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:25:53.921705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:53.921705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:53.921705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:53.921705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:53.922302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:53.922302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:53.922302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:53.922302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:53.922302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:53.922302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:53.922302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:53.922302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:53.922302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:53.922302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:53.922302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:53.922302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:53.922302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:53.922302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:53.922971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:53.922971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:53.922971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.922971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:53.923352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.923352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:53.923352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:53.923352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:53.923352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:53.923352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:53.923352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.923352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:53.923352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:53.923352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.923352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:53.923352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:53.923352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.923352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:53.923352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:53.923352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.923352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:25:53.924089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:25:53.924089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:53.924089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:53.924089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:53.924089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:53.924089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:53.924089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:53.924089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:53.924089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:53.924089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:53.924089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:53.924089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:53.924089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:53.924089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:53.924089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:53.924089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:53.924089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:53.924089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:53.924089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:53.924635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:53.924635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:53.924635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.924635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:53.925508 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:53.925817 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:53.925928 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:53.926120 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:53.926549 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:53.926962 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:53.927182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.927989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:53.927989 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:53.927989 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:53.927989 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:53.927989 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:53.927989 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:53.927989 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.927989 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:53.929027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.929027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:53.929027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:53.929027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:53.929027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.929027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:25:53.929027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:53.929027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:53.929027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:53.929027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:53.929027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:53.929027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:53.929358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:53.929358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:53.929358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:53.929358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:53.929358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:53.929358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:53.929358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:53.929358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:53.929358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:53.929358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:53.929358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:53.929358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:53.929358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:53.929699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.929699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:53.930061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.930061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:53.930061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:53.930061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:53.930061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:53.930061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:53.930061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.930061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:53.930061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:53.930061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.930061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:53.930061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:53.930061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.930061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:53.930061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:53.930061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:53.930783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:25:53.930783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:53.930783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:53.930783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:53.930783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:53.930783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:53.930783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:53.930783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:53.930783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:53.930783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:53.930783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:53.930783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:53.930783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:53.930783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:53.930783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:53.930783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:53.930783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:53.930783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:53.930783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:53.931261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:53.931261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:53.931261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:58.608711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:58.609603 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:58.609603 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:58.610128 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:58.642609 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:58.643296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.644448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:58.644448 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:58.644448 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:58.644448 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:58.644448 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:58.644448 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:58.644448 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.644448 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:58.645404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.645404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:58.645404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:58.645404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:58.645404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.645404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:25:58.645404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:58.645404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:58.645404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:58.645404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:58.645404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:58.645404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:58.645727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:58.645727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:58.645727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:58.645727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:58.645727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:58.645727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:58.645727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:58.645727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:58.645727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:58.645727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:58.645727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:58.645727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:58.645727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:58.646015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.646015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:58.646502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.646502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:58.646502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:58.646502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:58.646502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:58.646502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:58.646502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.646502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:58.646502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:58.646502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.646502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:58.646502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:58.646502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.646502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:58.646502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:58.646502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.646981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:25:58.646981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:58.646981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:58.646981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:58.646981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:58.646981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:58.646981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:58.646981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:58.646981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:58.646981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:58.646981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:58.646981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:58.646981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:58.646981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:58.646981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:58.646981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:58.646981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:58.646981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:58.646981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:58.647384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:58.647384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.647384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:58.842088 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:58.842350 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:58.842474 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:58.843419 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:58.843706 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:58.843967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.844607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:58.844607 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:58.844607 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:58.844607 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:58.844607 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:58.844607 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:58.844607 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.844607 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:58.845592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.845592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:58.845592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:58.845592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:58.845592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.845592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:25:58.845592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:58.845592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:58.845592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:58.845592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:58.845592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:58.845592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:58.846214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:58.846214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:58.846214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:58.846214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:58.846214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:58.846214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:58.846214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:58.846214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:58.846214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:58.846214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:58.846214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:58.846214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:58.846214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:58.846880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.846880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:58.846880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.846880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:58.846880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:58.846880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:58.846880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:58.846880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:58.846880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.846880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:58.846880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:58.846880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.846880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:58.846880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:58.846880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.847619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:58.847619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:58.847619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.847619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:25:58.847619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:58.847619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:58.847619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:58.847619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:58.847619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:58.847619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:58.847619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:58.847619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:58.847619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:58.847619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:58.847619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:58.847619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:58.847619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:58.847619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:58.847619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:58.848516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:58.848516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:58.848516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:58.848516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:58.848516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.848516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:58.898024 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:58.898218 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:58.898306 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:58.898394 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:25:58.899504 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:58.899928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.900720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:58.900720 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:58.900720 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:58.900720 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:58.900720 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:58.900720 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:58.900720 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.900720 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:58.901365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.901365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:25:58.901365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:58.901365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:58.901365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:58.901365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:58.901365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:58.901365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:58.901365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:58.901365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:58.901365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:58.901365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:25:58.901365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:58.901365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:25:58.901365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:25:58.901863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:58.901863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:25:58.901863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:58.901863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:25:58.901863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:25:58.901863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:58.901863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:25:58.901863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.901863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:25:58.902142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.902142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:58.902142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:58.902142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:58.902142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.902142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:25:58.902142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:25:58.902142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:25:58.902142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:58.902142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:58.902142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:58.902142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:58.902531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:58.902531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:58.902531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:58.902531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:58.902531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:58.902531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:58.902531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:58.902531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:58.902531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:58.902531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:58.902531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:58.902531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:58.902531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:58.902531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:58.902922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:58.902922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:58.902922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.902922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:58.902922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.902922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:58.902922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:58.902922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:58.902922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:58.902922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:58.902922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.902922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:58.902922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:58.902922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.903312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:58.903312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:58.903312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.903312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:58.903312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:58.903312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.903312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:25:58.903312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:25:58.903312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:25:58.903312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:58.903312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:58.903312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:58.903312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:58.903312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:58.903312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:58.903312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:58.903312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:58.903312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:58.903312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:58.904131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:58.904131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:58.904131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:58.904131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:58.904131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:58.904131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:58.904131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:58.904131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:58.904131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:58.904131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:58.904131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.904131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:58.917241 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:58.918018 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:58.918282 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:58.918573 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:58.919089 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:58.919327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.920087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:58.920087 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:58.920087 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:58.920087 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:58.920087 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:58.920087 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:58.920087 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.920087 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:58.921418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.921418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:25:58.921418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:58.921418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:58.921418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:58.921418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:58.921418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:58.921418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:58.921418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:58.921418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:25:58.921418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:58.921418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:25:58.921418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:58.921418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:25:58.921418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:25:58.922307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:58.922307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:25:58.922307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:58.922307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:25:58.922307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:25:58.922307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:58.922307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:25:58.922307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.922307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:25:58.922307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.922307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:58.922307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:58.922307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:58.922307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.923083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:25:58.923083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:25:58.923083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:58.923083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:58.923083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:58.923083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:58.923083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:58.923083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:58.923083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:58.923083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:58.923083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:58.923083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:58.923083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:58.923717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:58.923717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:58.923717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:58.923717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:58.923717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:58.923717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:58.923717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:58.923717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:58.923717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:58.923717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.923717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:58.923717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.924431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:58.924431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:58.924431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:58.924431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:58.924431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:58.924431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.924431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:58.924431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:58.924431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.924431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:58.924431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:58.924431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.924431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:58.924431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:58.924431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.924431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:25:58.924431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:25:58.924431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:25:58.924431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:25:58.925461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:58.925461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:58.925461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:58.925461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:58.925461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:58.925461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:58.925461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:58.925461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:58.925461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:58.925461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:58.925461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:58.925461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:58.925461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:58.925461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:58.925461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:58.925461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:58.925461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:58.925461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:58.925461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.925461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:25:58.926402 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:25:58.926402 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:25:58.926402 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:25:58.926402 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:25:58.926664 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:58.926946 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:25:58.927133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.927841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:25:58.927841 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:58.927841 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:25:58.927841 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:25:58.927841 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:58.927841 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:25:58.927841 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.927841 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:25:58.928702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.928702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:25:58.928702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:58.928702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:25:58.928702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.928702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:25:58.928702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:58.928702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:58.928702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:58.928702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:58.928702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:58.928702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:58.929306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:25:58.929306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:58.929306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:25:58.929306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:58.929306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:25:58.929306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:25:58.929306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:58.929306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:25:58.929306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:58.929306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:25:58.929306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:25:58.929306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:58.929306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:25:58.929919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.929919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:25:58.929919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.929919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:25:58.929919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:25:58.929919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:25:58.929919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:25:58.929919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:25:58.929919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.929919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:25:58.929919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:25:58.929919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.929919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:25:58.929919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:25:58.929919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.930638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:25:58.930638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:25:58.930638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:25:58.930638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:25:58.930638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:25:58.930638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:58.930638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:58.930638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:25:58.930638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:25:58.930638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:25:58.930638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:25:58.930638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:25:58.930638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:25:58.930638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:25:58.930638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:25:58.930638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:25:58.930638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:25:58.930638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:25:58.930638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:25:58.931541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:25:58.931541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:25:58.931541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:25:58.931541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:25:58.931541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:25:58.931541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:03.609302 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:03.610010 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:03.610010 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:03.611557 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:03.618859 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=241 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:26:03.648017 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:03.648275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.649032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:03.649032 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:03.649032 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:03.649032 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:03.649032 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:03.649032 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:03.649032 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.649032 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:03.650346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.650346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:03.650346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:03.650346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:03.650346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.650346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:26:03.650346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:03.650346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:03.650346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:03.650346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:03.650346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:03.650346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:03.651313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:03.651313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:03.651313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:03.651313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:03.651313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:03.651313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:03.651313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:03.651313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:03.651313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:03.651313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:03.651313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:03.651313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:03.651313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:03.652034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.652034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:03.652034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.652034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:03.652034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:03.652034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:03.652034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:03.652034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:03.652034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.652034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:03.652034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:03.652034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.652034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:03.652034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:03.652034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.652526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:03.652526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:03.652526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.652526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:26:03.652526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:03.652526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:03.652526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:03.652526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:03.652526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:03.652526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:03.652526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:03.652526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:03.652526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:03.652526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:03.652526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:03.652526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:03.652526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:03.652526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:03.652526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:03.653143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:03.653143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:03.653143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:03.653143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:03.653143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.653143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:03.842815 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:03.843068 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:03.843252 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:03.844445 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:03.844757 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:03.844944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.845654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:03.845654 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:03.845654 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:03.845654 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:03.845654 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:03.845654 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:03.845654 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.845654 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:03.847270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.847270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:03.847270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:03.847270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:03.847270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.847270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:26:03.847270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:03.847270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:03.847270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:03.847270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:03.847270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:03.847270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:03.847951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:03.847951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:03.847951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:03.847951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:03.847951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:03.847951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:03.847951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:03.847951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:03.847951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:03.847951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:03.847951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:03.847951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:03.847951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:03.848667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.848667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:03.849586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.849586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:03.849586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:03.849586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:03.849586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:03.849586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:03.849586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.849586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:03.849586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:03.849586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.849586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:03.849586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:03.849586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.849586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:03.849586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:03.849586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.850429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:26:03.850429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:03.850429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:03.850429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:03.850429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:03.850429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:03.850429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:03.850429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:03.850429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:03.850429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:03.850429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:03.850429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:03.850429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:03.850429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:03.850429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:03.850429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:03.850429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:03.850429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:03.850429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:03.851393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:03.851393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.851393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:03.898556 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:03.899018 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:03.899237 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:03.899458 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:26:03.900246 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:03.900479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.901245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:03.901245 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:03.901245 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:03.901245 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:03.901245 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:03.901245 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:03.901245 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.901245 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:03.902494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.902494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:26:03.902494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:03.902494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:03.902494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:03.902494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:03.902494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:03.902494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:03.902494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:03.902494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:03.902494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:03.902494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:26:03.902494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:03.902494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:26:03.902494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:26:03.903339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:03.903339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:26:03.903339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:03.903339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:26:03.903339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:26:03.903339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:03.903339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:26:03.903339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.903339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:26:03.903838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.903838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:03.903838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:03.903838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:03.903838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.903838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:26:03.903838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:26:03.903838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:26:03.903838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:03.903838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:03.903838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:03.903838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:03.904537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:03.904537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:03.904537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:03.904537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:03.904537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:03.904537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:03.904537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:03.904537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:03.904537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:03.904537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:03.904537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:03.904537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:03.904537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:03.904537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:03.905152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:03.905152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:03.905152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.905152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:03.905633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.905633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:03.905633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:03.905633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:03.905633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:03.905633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:03.905633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.905633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:03.905633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:03.905633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.905633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:03.905633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:03.905633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.905633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:03.905633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:03.905633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.906324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:26:03.906324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:26:03.906324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:26:03.906324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:03.906324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:03.906324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:03.906324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:03.906324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:03.906324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:03.906324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:03.906324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:03.906324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:03.906324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:03.906324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:03.906324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:03.906324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:03.906324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:03.906324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:03.907120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:03.907120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:03.907120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:03.907120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:03.907120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:03.907120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.907120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:03.917707 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:03.918033 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:03.918211 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:03.918412 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:03.918924 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:03.919320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.920216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:03.920216 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:03.920216 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:03.920216 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:03.920216 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:03.920216 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:03.920216 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.920216 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:03.921397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.921397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:26:03.921397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:03.921397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:03.921397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:03.921397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:03.921397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:03.921397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:03.921397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:03.921397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:03.921397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:03.921397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:26:03.921397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:03.921397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:26:03.921397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:26:03.922045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:03.922045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:26:03.922045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:03.922045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:26:03.922045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:26:03.922045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:03.922045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:26:03.922045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.922045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:26:03.922981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.922981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:03.922981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:03.922981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:03.922981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.922981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:26:03.922981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:26:03.922981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:03.922981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:03.922981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:03.922981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:03.923495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:03.923495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:03.923495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:03.923495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:03.923495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:03.923495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:03.923495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:03.923495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:03.923495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:03.923495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:03.923495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:03.923495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:03.923495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:03.923495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:03.924156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:03.924156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:03.924156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.924156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:03.924513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.924513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:03.924513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:03.924513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:03.924513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:03.924513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:03.924513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.924513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:03.924513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:03.924513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.924513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:03.924513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:03.924513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.924513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:03.924513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:03.924513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.924513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:26:03.925235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:26:03.925235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:03.925235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:03.925235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:03.925235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:03.925235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:03.925235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:03.925235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:03.925235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:03.925235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:03.925235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:03.925235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:03.925235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:03.925235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:03.925235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:03.925235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:03.925235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:03.925235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:03.925235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:03.925714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:03.925714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:03.925714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.925714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:03.926121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:03.926292 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:03.926396 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:03.926541 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:03.927052 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:03.927441 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:03.927749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.928432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:03.928432 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:03.928432 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:03.928432 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:03.928432 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:03.928432 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:03.928432 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.928432 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:03.929264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.929264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:03.929264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:03.929264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:03.929264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.929264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:26:03.929264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:03.929264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:03.929264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:03.929264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:03.929264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:03.929264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:03.929629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:03.929629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:03.929629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:03.929629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:03.929629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:03.929629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:03.929629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:03.929629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:03.929629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:03.929629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:03.929629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:03.929629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:03.929629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:03.929990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.929990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:03.930216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.930216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:03.930216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:03.930216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:03.930216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:03.930216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:03.930216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.930216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:03.930216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:03.930216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.930216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:03.930216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:03.930216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.930216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:03.930216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:03.930216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:03.930639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:26:03.930639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:03.930639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:03.930639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:03.930639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:03.930639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:03.930639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:03.930639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:03.930639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:03.930639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:03.930639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:03.930639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:03.930639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:03.930639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:03.930639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:03.930639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:03.930639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:03.930639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:03.930639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:03.931287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:03.931287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:03.931287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:08.609989 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:08.610868 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:08.610868 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:08.611271 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:08.647779 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:08.648153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.648984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:08.648984 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:08.648984 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:08.648984 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:08.648984 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:08.648984 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:08.648984 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.648984 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:08.649761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.649761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:08.649761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:08.649761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:08.649761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.649761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:26:08.649761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:08.649761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:08.649761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:08.649761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:08.649761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:08.649761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:08.650382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:08.650382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:08.650382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:08.650382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:08.650382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:08.650382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:08.650382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:08.650382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:08.650382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:08.650382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:08.650382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:08.650382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:08.650382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:08.651083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.651083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:08.651194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.651194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:08.651194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:08.651194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:08.651194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:08.651194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:08.651194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.651194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:08.651194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:08.651194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.651194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:08.651194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:08.651194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.651194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:08.651194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:08.651194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.652031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:26:08.652031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:08.652031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:08.652031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:08.652031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:08.652031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:08.652031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:08.652031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:08.652031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:08.652031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:08.652031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:08.652031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:08.652031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:08.652031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:08.652031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:08.652031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:08.652031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:08.652031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:08.652031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:08.652936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:08.652936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.652936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:08.843458 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:08.843863 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:08.844039 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:08.845050 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:08.845270 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:08.845407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.846138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:08.846138 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:08.846138 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:08.846138 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:08.846138 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:08.846138 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:08.846138 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.846138 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:08.847953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.847953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:08.847953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:08.847953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:08.847953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.847953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:26:08.847953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:08.847953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:08.847953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:08.847953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:08.847953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:08.847953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:08.848446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:08.848446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:08.848446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:08.848446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:08.848446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:08.848446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:08.848446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:08.848446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:08.848446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:08.848446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:08.848446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:08.848446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:08.848446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:08.848856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.848856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:08.852019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.852019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:08.852019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:08.852019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:08.852019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:08.852019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:08.852019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.852019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:08.852019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:08.852019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.852019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:08.852019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:08.852019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.852019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:08.852019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:08.852019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.852857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:26:08.852857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:08.852857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:08.852857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:08.852857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:08.852857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:08.852857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:08.852857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:08.852857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:08.852857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:08.852857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:08.852857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:08.852857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:08.852857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:08.852857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:08.852857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:08.852857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:08.852857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:08.852857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:08.853591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:08.853591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.853591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:08.899250 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:08.899881 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:08.900024 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:08.900235 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:26:08.900888 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:08.901093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.901807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:08.901807 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:08.901807 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:08.901807 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:08.901807 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:08.901807 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:08.901807 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.901807 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:08.903194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.903194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:26:08.903194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:08.903194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:08.903194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:08.903194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:08.903194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:08.903194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:08.903194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:08.903194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:08.903194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:08.903194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:26:08.903194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:08.903194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:26:08.903194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:26:08.904051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:08.904051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:26:08.904051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:08.904051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:26:08.904051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:26:08.904051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:08.904051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:26:08.904051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.904051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:26:08.904498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.904498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:08.904498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:08.904498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:08.904498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.904498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:26:08.904498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:26:08.904498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:26:08.904498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:08.904498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:08.904498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:08.904498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:08.905139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:08.905139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:08.905139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:08.905139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:08.905139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:08.905139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:08.905139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:08.905139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:08.905139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:08.905139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:08.905139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:08.905139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:08.905139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:08.905139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:08.906010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:08.906010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:08.906010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.906010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:08.906207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.906207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:08.906207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:08.906207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:08.906207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:08.906207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:08.906207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.906207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:08.906207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:08.906207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.906207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:08.906207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:08.906207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.906207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:08.906207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:08.906207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.907237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:26:08.907237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:26:08.907237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:26:08.907237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:08.907237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:08.907237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:08.907237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:08.907237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:08.907237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:08.907237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:08.907237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:08.907237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:08.907237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:08.907237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:08.907237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:08.907237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:08.907237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:08.907237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:08.907733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:08.907733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:08.907733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:08.907733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:08.907733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:08.907733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.907733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:08.918205 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:08.918595 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:08.918858 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:08.919065 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:08.919471 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:08.919665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.920164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:08.920164 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:08.920164 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:08.920164 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:08.920164 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:08.920164 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:08.920164 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.920164 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:08.920916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.920916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:26:08.920916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:08.920916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:08.920916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:08.920916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:08.920916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:08.920916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:08.920916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:08.920916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:08.920916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:08.920916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:26:08.920916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:08.920916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:26:08.920916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:26:08.921545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:08.921545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:26:08.921545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:08.921545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:26:08.921545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:26:08.921545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:08.921545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:26:08.921545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.921545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:26:08.922299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.922299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:08.922299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:08.922299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:08.922299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.922299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:26:08.922299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:26:08.922299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:08.922299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:08.922299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:08.922299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:08.922873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:08.922873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:08.922873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:08.922873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:08.922873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:08.922873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:08.922873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:08.922873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:08.922873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:08.922873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:08.922873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:08.922873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:08.922873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:08.922873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:08.923504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:08.923504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:08.923504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.923504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:08.923504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.923504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:08.923504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:08.923504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:08.923504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:08.923504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:08.923504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.923504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:08.923504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:08.923504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.924263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:08.924263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:08.924263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.924263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:08.924263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:08.924263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.924263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:26:08.924263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:26:08.924263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:08.924263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:08.924263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:08.924263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:08.924263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:08.924263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:08.924263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:08.924263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:08.924263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:08.924263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:08.924263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:08.925247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:08.925247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:08.925247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:08.925247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:08.925247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:08.925247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:08.925247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:08.925247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:08.925247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:08.925247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.925247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:08.926666 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:08.927072 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:08.927188 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:08.927407 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:08.928140 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:08.928558 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:08.928880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.929671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:08.929671 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:08.929671 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:08.929671 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:08.929671 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:08.929671 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:08.929671 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.929671 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:08.930576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.930576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:08.930576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:08.930576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:08.930576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.930576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:26:08.930576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:08.930576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:08.930576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:08.930576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:08.930576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:08.930576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:08.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:08.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:08.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:08.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:08.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:08.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:08.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:08.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:08.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:08.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:08.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:08.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:08.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:08.932185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.932185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:08.932185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.932185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:08.932185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:08.932185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:08.932185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:08.932185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:08.932185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.932185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:08.932185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:08.932185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.932185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:08.932185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:08.932185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.933017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:08.933017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:08.933017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:08.933017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:26:08.933017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:08.933017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:08.933017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:08.933017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:08.933017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:08.933017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:08.933017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:08.933017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:08.933017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:08.933017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:08.933017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:08.933017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:08.933017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:08.933017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:08.933017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:08.934292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:08.934292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:08.934292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:08.934292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:08.934292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:08.934292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:13.055008 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:13.055922 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:13.056021 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:13.066298 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
Apr 28 20:26:13.066412 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=106506) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
Apr 28 20:26:13.066601 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 7ff08466-af6d-4e13-bb1d-8987170a5734 (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:26:13.100940 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-7ff08466-af6d-4e13-bb1d-8987170a5734, resource={'id': '7ff08466-af6d-4e13-bb1d-8987170a5734', 'name': 'private', 'tenant_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'project_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['4b461bd9-09ea-4f87-a007-a11ab022ff7f', '8f3677a2-e2be-443a-a90c-2cd7b3095c49'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-28T20:10:58Z', 'updated_at': '2026-04-28T20:11:02Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 46342}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.101455 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-7ff08466-af6d-4e13-bb1d-8987170a5734, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.102159 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-27fbc209-4b05-4b4a-8d5c-40d6d87b7e22, external_ids_update=None, columns={'tag': 46342}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.102393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.103350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:13.103350 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.103350 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:13.103350 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.103350 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.103350 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:13.103350 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.103350 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:13.104424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 7ff08466-af6d-4e13-bb1d-8987170a5734 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.104424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:13.104424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:26:13.104424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:26:13.104424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:26:13.104424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:26:13.104424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:26:13.104424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:13.104424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.104424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:13.104424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:13.104995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:13.104995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.104995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:13.104995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:13.104995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:13.104995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:13.104995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:13.104995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:13.104995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:13.104995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:13.104995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.104995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:13.104995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.104995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.105654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:13.105654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.105654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:13.105654 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 8a1fb774-5693-4139-8b75-6ab5273228c7 (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:26:13.135863 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, resource={'id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'name': 'public', 'tenant_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'project_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['335d12ce-2d0c-4086-b3f4-2af390c03d70', '5308c10f-28d1-441c-acb2-d1d6dc3d2ece'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-04-28T20:11:05Z', 'updated_at': '2026-04-28T20:11:13Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.136379 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.137249 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-7dbf713e-d963-4884-ae64-da4ce196e719, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.137834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.138763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:13.138763 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.138763 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:13.138763 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.138763 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.138763 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:13.138763 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.138763 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:13.139774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 8a1fb774-5693-4139-8b75-6ab5273228c7 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.139774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:13.139774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:26:13.139774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:26:13.139774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:26:13.139774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:26:13.139774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:26:13.139774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:13.139774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.139774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:13.139774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:13.140415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:13.140415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.140415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:13.140415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:13.140415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:13.140415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:13.140415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:13.140415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:13.140415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:13.140415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:13.140415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.140415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:13.140415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.140415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.141033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:13.141033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.141033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:13.141033 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource c6718637-daa9-4db1-bd11-63abe3891db6 (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:26:13.174992 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-c6718637-daa9-4db1-bd11-63abe3891db6, resource={'id': 'c6718637-daa9-4db1-bd11-63abe3891db6', 'name': 'admin_net', 'tenant_id': '3eb96b75a3044dd0a338616e1d9081d0', 'project_id': '3eb96b75a3044dd0a338616e1d9081d0', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['c1cc9883-a7e2-4abc-8892-a7c1b200d3ad'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-28T20:15:13Z', 'updated_at': '2026-04-28T20:15:14Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 7626}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.176523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-c6718637-daa9-4db1-bd11-63abe3891db6, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.177183 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-7461d8df-a8a0-40c3-87a1-26ed88f9f7ce, external_ids_update=None, columns={'tag': 7626}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.177380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.178280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:13.178280 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.178280 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:13.178280 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.178280 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.178280 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:13.178280 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.178280 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:13.179451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource c6718637-daa9-4db1-bd11-63abe3891db6 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.179451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:13.179451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:26:13.179451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:26:13.179451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:26:13.179451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:26:13.179451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:26:13.179451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:13.179451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.179451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:13.179451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:13.180008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:13.180008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.180008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:13.180008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:13.180008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:13.180008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:13.180008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:13.180008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:13.180008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:13.180008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:13.180008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.180008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:13.180008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.180008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.180602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:13.180602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.180602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:13.180602 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource d3dfebbd-7943-4813-b509-739084ada63b (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:26:13.210765 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d3dfebbd-7943-4813-b509-739084ada63b, resource={'id': 'd3dfebbd-7943-4813-b509-739084ada63b', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-pi48i', 'tenant_id': 'a11fcd8844504e7eaf509ece683e8c8f', 'project_id': 'a11fcd8844504e7eaf509ece683e8c8f', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['c68bc19b-c88a-4cd2-bf3a-d847d267d23f'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-28T20:15:49Z', 'updated_at': '2026-04-28T20:15:50Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 58143}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.211167 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-d3dfebbd-7943-4813-b509-739084ada63b, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-pi48i', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.211717 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-ed33b0a1-b0b7-4e33-aa96-92c114a82f5d, external_ids_update=None, columns={'tag': 58143}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.211908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.212843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:13.212843 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.212843 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:13.212843 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.212843 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.212843 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:13.212843 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.212843 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:13.214173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource d3dfebbd-7943-4813-b509-739084ada63b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.214173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:13.214173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:26:13.214173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:26:13.214173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:26:13.214173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:26:13.214173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:26:13.214173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:13.214173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.214173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:13.214173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:13.214765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:13.214765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.214765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:13.214765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:13.214765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:13.214765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:13.214765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:13.214765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:13.214765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:13.214765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:13.214765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.214765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:13.214765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.214765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.215378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:13.215378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.215378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:13.215378 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 6feb526a-13c6-41ac-983d-ea1d2251ff3d (type: routers) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:26:13.312623 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, resource={'id': '6feb526a-13c6-41ac-983d-ea1d2251ff3d', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-pi48i', 'project_id': 'a11fcd8844504e7eaf509ece683e8c8f', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'external_fixed_ips': [{'subnet_id': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece', 'ip_address': '172.24.5.87'}, {'subnet_id': '335d12ce-2d0c-4086-b3f4-2af390c03d70', 'ip_address': '2001:db8::2d8'}], 'enable_snat': True}, 'gw_port_id': '73493630-b0ac-48ec-be98-5b048d2e2ce7', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-pi48i', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'external_fixed_ips': [{'ip_address': '172.24.5.87', 'subnet_id': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece'}, {'ip_address': '2001:db8::2d8', 'subnet_id': '335d12ce-2d0c-4086-b3f4-2af390c03d70'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-04-28T20:15:51Z', 'updated_at': '2026-04-28T20:15:54Z', 'revision_number': 3, 'tenant_id': 'a11fcd8844504e7eaf509ece683e8c8f'}, resource_type=routers, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.313078 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-pi48i', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.313437 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "73493630-b0ac-48ec-be98-5b048d2e2ce7"')], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.313775 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "73493630-b0ac-48ec-be98-5b048d2e2ce7"')], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.314029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.315234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:13.315234 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.315234 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:13.315234 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.315234 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.315234 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:13.315234 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.315234 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:13.315470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Unable to update router 6feb526a-13c6-41ac-983d-ea1d2251ff3d. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.316991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 6feb526a-13c6-41ac-983d-ea1d2251ff3d (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.316991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:13.316991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:26:13.316991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:26:13.316991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:26:13.316991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:26:13.316991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
Apr 28 20:26:13.316991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Apr 28 20:26:13.316991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
Apr 28 20:26:13.316991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Apr 28 20:26:13.316991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Apr 28 20:26:13.317322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Apr 28 20:26:13.317322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
Apr 28 20:26:13.317322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:13.317322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.317322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:13.317322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:13.317322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:13.317322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.317322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:13.317322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:13.317322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:13.317322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:13.317322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:13.317322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:13.317724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:13.317724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:13.317724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.317724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:13.317724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.317724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.317724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:13.317724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.317724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:13.317724 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 73493630-b0ac-48ec-be98-5b048d2e2ce7 (type: router_ports) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:26:13.422271 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-73493630-b0ac-48ec-be98-5b048d2e2ce7, resource={'id': '73493630-b0ac-48ec-be98-5b048d2e2ce7', 'name': '', 'network_id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'project_id': '', 'mac_address': 'fa:16:3e:f8:07:38', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6feb526a-13c6-41ac-983d-ea1d2251ff3d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece', 'ip_address': '172.24.5.87'}, {'subnet_id': '335d12ce-2d0c-4086-b3f4-2af390c03d70', 'ip_address': '2001:db8::2d8'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000171185', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-04-28T20:15:51Z', 'updated_at': '2026-04-28T20:15:54Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.422701 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-73493630-b0ac-48ec-be98-5b048d2e2ce7, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece 335d12ce-2d0c-4086-b3f4-2af390c03d70', 'neutron:network_name': 'neutron-8a1fb774-5693-4139-8b75-6ab5273228c7', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.87/24', '2001:db8::2d8/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.423748 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=73493630-b0ac-48ec-be98-5b048d2e2ce7, lrouter_port=lrp-73493630-b0ac-48ec-be98-5b048d2e2ce7, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.424267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.425238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:13.425238 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.425238 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:13.425238 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.425238 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.425238 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:13.425238 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.425238 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:13.427349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 73493630-b0ac-48ec-be98-5b048d2e2ce7 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.427349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:13.427349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:26:13.427349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:26:13.427349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:26:13.427349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:26:13.427349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1971, in update_router_port
Apr 28 20:26:13.427349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:13.427349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.427349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:13.427349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:13.428185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:13.428185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.428185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:13.428185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:13.428185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:13.428185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:13.428185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:13.428185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:13.428185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:13.428185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:13.428185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.428185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:13.428185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.428185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.429969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:13.429969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.429969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:13.429969 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_for_inconsistencies finished in 0.371 seconds
Apr 28 20:26:13.610669 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:13.611251 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:13.611494 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:13.612885 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:13.646303 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.646690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.647557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:13.647557 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.647557 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:13.647557 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.647557 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.647557 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:13.647557 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.647557 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:13.648458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.648458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:13.648458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:13.648458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:13.648458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.648458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:26:13.648458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:13.648458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.648458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:13.648458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:13.648458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:13.648458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.648890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:13.648890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:13.648890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:13.648890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:13.648890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:13.648890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:13.648890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:13.648890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:13.648890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.648890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:13.648890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.648890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.648890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:13.649320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.649320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:13.649581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.649581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:13.649581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:13.649581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:13.649581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:13.649581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:13.649581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.649581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:13.649581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:13.649581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.649581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:13.649581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:13.649581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.649581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:13.649581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:13.649581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.649960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:26:13.649960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:13.649960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.649960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:13.649960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:13.649960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:13.649960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.649960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:13.649960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:13.649960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:13.649960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:13.649960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:13.649960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:13.649960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:13.649960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:13.649960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.649960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:13.649960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.649960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.650344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:13.650344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.650344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:13.844123 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:13.844828 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:13.845089 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:13.846403 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.846623 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.846862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.847721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:13.847721 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.847721 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:13.847721 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.847721 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.847721 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:13.847721 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.847721 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:13.849670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.849670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:13.849670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:13.849670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:13.849670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.849670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:26:13.849670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:13.849670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.849670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:13.849670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:13.849670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:13.849670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.850537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:13.850537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:13.850537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:13.850537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:13.850537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:13.850537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:13.850537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:13.850537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:13.850537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.850537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:13.850537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.850537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.850537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:13.851109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.851109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:13.851418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.851418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:13.851418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:13.851418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:13.851418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:13.851418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:13.851418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.851418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:13.851418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:13.851418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.851418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:13.851418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:13.851418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.851418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:13.851418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:13.851418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.851895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:26:13.851895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:13.851895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.851895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:13.851895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:13.851895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:13.851895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.851895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:13.851895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:13.851895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:13.851895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:13.851895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:13.851895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:13.851895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:13.851895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:13.851895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.851895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:13.851895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.851895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.852395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:13.852395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.852395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:13.900001 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:13.900345 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:13.900540 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:13.900760 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:26:13.901470 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.901714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.902409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:13.902409 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.902409 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:13.902409 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.902409 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.902409 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:13.902409 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.902409 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:13.903919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.903919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:26:13.903919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:13.903919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:13.903919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.903919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:13.903919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:13.903919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:13.903919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.903919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:13.903919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:13.903919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:26:13.903919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:13.903919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:26:13.903919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:26:13.904573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:13.904573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:26:13.904573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.904573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:26:13.904573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.904573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.904573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:26:13.904573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.904573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:26:13.905301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.905301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:13.905301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:13.905301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:13.905301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.905301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:26:13.905301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:26:13.905301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:26:13.905301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:13.905301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:13.905301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.905301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:13.905918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:13.905918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:13.905918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.905918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:13.905918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:13.905918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:13.905918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:13.905918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:13.905918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:13.905918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:13.905918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:13.905918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.905918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:13.905918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.906519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.906519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:13.906519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.906519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:13.907124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.907124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:13.907124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:13.907124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:13.907124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:13.907124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:13.907124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.907124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:13.907124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:13.907124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.907124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:13.907124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:13.907124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.907124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:13.907124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:13.907124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.907866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:26:13.907866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:26:13.907866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:26:13.907866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:13.907866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:13.907866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.907866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:13.907866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:13.907866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:13.907866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.907866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:13.907866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:13.907866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:13.907866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:13.907866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:13.907866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:13.907866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:13.907866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:13.908631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.908631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:13.908631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.908631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.908631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:13.908631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.908631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:13.918708 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:13.919143 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:13.919342 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:13.919550 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:13.920117 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.920538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.921424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:13.921424 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.921424 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:13.921424 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.921424 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.921424 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:13.921424 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.921424 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:13.922907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.922907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:26:13.922907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:13.922907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:13.922907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.922907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:13.922907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:13.922907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:13.922907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.922907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:13.922907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:13.922907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:26:13.922907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:13.922907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:26:13.922907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:26:13.923866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:13.923866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:26:13.923866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.923866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:26:13.923866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.923866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.923866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:26:13.923866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.923866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:26:13.924414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.924414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:13.924414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:13.924414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:13.924414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.924414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:26:13.924414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:26:13.924414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:13.924414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:13.924414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.924414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:13.924935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:13.924935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:13.924935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.924935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:13.924935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:13.924935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:13.924935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:13.924935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:13.924935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:13.924935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:13.924935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:13.924935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.924935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:13.924935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.925609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.925609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:13.925609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.925609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:13.926170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.926170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:13.926170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:13.926170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:13.926170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:13.926170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:13.926170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.926170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:13.926170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:13.926170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.926170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:13.926170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:13.926170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.926170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:13.926170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:13.926170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.926170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:26:13.926919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:26:13.926919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:13.926919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:13.926919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.926919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:13.926919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:13.926919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:13.926919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.926919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:13.926919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:13.926919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:13.926919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:13.926919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:13.926919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:13.926919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:13.926919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:13.926919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.926919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:13.926919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.927810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.927810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:13.927810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.927810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:13.927810 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:13.927810 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:13.928116 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:13.928171 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:13.928730 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.929096 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:13.929349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.930173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:13.930173 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.930173 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:13.930173 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.930173 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.930173 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:13.930173 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.930173 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:13.930985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.930985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:13.930985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:13.930985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:13.930985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.930985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:26:13.930985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:13.930985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.930985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:13.930985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:13.930985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:13.930985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.931364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:13.931364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:13.931364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:13.931364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:13.931364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:13.931364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:13.931364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:13.931364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:13.931364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.931364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:13.931364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.931364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.931364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:13.931757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.931757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:13.932034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.932034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:13.932034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:13.932034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:13.932034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:13.932034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:13.932034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.932034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:13.932034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:13.932034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.932034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:13.932034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:13.932034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.932034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:13.932034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:13.932034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:13.932481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:26:13.932481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:13.932481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.932481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:13.932481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:13.932481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:13.932481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:13.932481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:13.932481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:13.932481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:13.932481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:13.932481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:13.932481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:13.932481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:13.932481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:13.932481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:13.932481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:13.932481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:13.932481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:13.933026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:13.933026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:13.933026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:18.611383 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:18.612020 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:18.612213 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:18.613613 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:18.650476 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:18.650776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.651430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:18.651430 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:18.651430 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:18.651430 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:18.651430 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:18.651430 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:18.651430 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.651430 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:18.652184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.652184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:18.652184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:18.652184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:18.652184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.652184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:26:18.652184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:18.652184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:18.652184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:18.652184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:18.652184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:18.652184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:18.652846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:18.652846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:18.652846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:18.652846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:18.652846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:18.652846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:18.652846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:18.652846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:18.652846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:18.652846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:18.652846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:18.652846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:18.652846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:18.653433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.653433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:18.653433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.653433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:18.653433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:18.653433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:18.653433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:18.653433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:18.653433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.653433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:18.653433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:18.653433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.653433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:18.653433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:18.653433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.654185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:18.654185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:18.654185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.654185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:26:18.654185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:18.654185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:18.654185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:18.654185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:18.654185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:18.654185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:18.654185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:18.654185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:18.654185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:18.654185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:18.654185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:18.654185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:18.654185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:18.654185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:18.654185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:18.655290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:18.655290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:18.655290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:18.655290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:18.655290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.655290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:18.844768 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:18.845323 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:18.845543 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:18.846586 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:18.846878 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:18.847050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.847750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:18.847750 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:18.847750 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:18.847750 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:18.847750 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:18.847750 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:18.847750 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.847750 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:18.849268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.849268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:18.849268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:18.849268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:18.849268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.849268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:26:18.849268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:18.849268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:18.849268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:18.849268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:18.849268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:18.849268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:18.849940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:18.849940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:18.849940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:18.849940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:18.849940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:18.849940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:18.849940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:18.849940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:18.849940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:18.849940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:18.849940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:18.849940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:18.849940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:18.850583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.850583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:18.851153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.851153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:18.851153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:18.851153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:18.851153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:18.851153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:18.851153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.851153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:18.851153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:18.851153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.851153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:18.851153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:18.851153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.851153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:18.851153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:18.851153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.852031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:26:18.852031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:18.852031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:18.852031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:18.852031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:18.852031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:18.852031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:18.852031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:18.852031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:18.852031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:18.852031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:18.852031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:18.852031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:18.852031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:18.852031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:18.852031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:18.852031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:18.852031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:18.852031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:18.852973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:18.852973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.852973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:18.900508 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:18.900910 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:18.901092 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:18.901273 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:26:18.901932 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:18.902249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.903140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:18.903140 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:18.903140 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:18.903140 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:18.903140 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:18.903140 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:18.903140 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.903140 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:18.904399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.904399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:26:18.904399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:18.904399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:18.904399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:18.904399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:18.904399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:18.904399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:18.904399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:18.904399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:18.904399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:18.904399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:26:18.904399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:18.904399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:26:18.904399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:26:18.905135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:18.905135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:26:18.905135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:18.905135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:26:18.905135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:26:18.905135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:18.905135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:26:18.905135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.905135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:26:18.905741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.905741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:18.905741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:18.905741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:18.905741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.905741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:26:18.905741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:26:18.905741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:26:18.905741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:18.905741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:18.905741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:18.905741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:18.906308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:18.906308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:18.906308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:18.906308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:18.906308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:18.906308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:18.906308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:18.906308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:18.906308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:18.906308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:18.906308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:18.906308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:18.906308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:18.906308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:18.906923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:18.906923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:18.906923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.906923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:18.907355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.907355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:18.907355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:18.907355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:18.907355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:18.907355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:18.907355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.907355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:18.907355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:18.907355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.907355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:18.907355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:18.907355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.907355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:18.907355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:18.907355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.907918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:26:18.907918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:26:18.907918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:26:18.907918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:18.907918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:18.907918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:18.907918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:18.907918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:18.907918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:18.907918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:18.907918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:18.907918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:18.907918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:18.907918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:18.907918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:18.907918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:18.907918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:18.907918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:18.908435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:18.908435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:18.908435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:18.908435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:18.908435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:18.908435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.908435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:18.919177 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:18.919464 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:18.919555 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:18.919697 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:18.920241 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:18.920595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.921336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:18.921336 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:18.921336 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:18.921336 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:18.921336 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:18.921336 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:18.921336 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.921336 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:18.922067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.922067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:26:18.922067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:18.922067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:18.922067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:18.922067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:18.922067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:18.922067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:18.922067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:18.922067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:18.922067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:18.922067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:26:18.922067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:18.922067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:26:18.922067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:26:18.922922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:18.922922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:26:18.922922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:18.922922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:26:18.922922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:26:18.922922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:18.922922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:26:18.922922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.922922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:26:18.922922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.922922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:18.922922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:18.922922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:18.922922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.923793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:26:18.923793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:26:18.923793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:18.923793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:18.923793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:18.923793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:18.923793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:18.923793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:18.923793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:18.923793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:18.923793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:18.923793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:18.923793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:18.924456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:18.924456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:18.924456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:18.924456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:18.924456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:18.924456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:18.924456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:18.924456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:18.924456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:18.924456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.924456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:18.924456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.924854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:18.924854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:18.924854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:18.924854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:18.924854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:18.924854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.924854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:18.924854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:18.924854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.924854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:18.924854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:18.924854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.924854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:18.924854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:18.924854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.924854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:26:18.924854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:26:18.924854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:18.924854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:18.925437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:18.925437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:18.925437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:18.925437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:18.925437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:18.925437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:18.925437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:18.925437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:18.925437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:18.925437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:18.925437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:18.925437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:18.925437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:18.925437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:18.925437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:18.925437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:18.925437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:18.925437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:18.925437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.925437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:18.927771 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:18.928107 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:18.928292 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:18.928535 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:18.929031 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:18.929371 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:18.929593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.930332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:18.930332 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:18.930332 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:18.930332 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:18.930332 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:18.930332 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:18.930332 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.930332 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:18.931830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.931830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:18.931830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:18.931830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:18.931830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.931830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:26:18.931830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:18.931830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:18.931830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:18.931830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:18.931830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:18.931830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:18.932208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:18.932208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:18.932208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:18.932208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:18.932208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:18.932208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:18.932208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:18.932208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:18.932208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:18.932208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:18.932208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:18.932208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:18.932208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:18.932568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.932568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:18.933116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.933116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:18.933116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:18.933116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:18.933116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:18.933116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:18.933116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.933116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:18.933116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:18.933116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.933116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:18.933116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:18.933116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.933116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:18.933116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:18.933116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:18.933745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:26:18.933745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:18.933745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:18.933745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:18.933745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:18.933745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:18.933745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:18.933745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:18.933745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:18.933745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:18.933745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:18.933745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:18.933745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:18.933745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:18.933745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:18.933745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:18.933745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:18.933745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:18.933745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:18.934301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:18.934301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:18.934301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:23.612094 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:23.612982 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:23.613075 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:23.614449 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:23.663661 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:23.664072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.665003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:23.665003 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:23.665003 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:23.665003 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:23.665003 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:23.665003 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:23.665003 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.665003 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:23.666222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.052 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.666222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:23.666222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:23.666222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:23.666222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.666222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:26:23.666222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:23.666222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:23.666222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:23.666222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:23.666222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:23.666222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:23.667434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:23.667434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:23.667434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:23.667434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:23.667434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:23.667434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:23.667434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:23.667434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:23.667434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:23.667434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:23.667434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:23.667434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:23.667434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:23.668453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.668453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:23.668453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.668453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:23.668453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:23.668453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:23.668453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:23.668453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:23.668453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.668453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:23.668453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:23.668453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.668453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:23.668453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:23.668453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.670389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:23.670389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:23.670389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.670389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:26:23.670389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:23.670389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:23.670389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:23.670389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:23.670389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:23.670389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:23.670389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:23.670389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:23.670389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:23.670389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:23.670389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:23.670389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:23.670389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:23.670389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:23.670389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:23.671240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:23.671240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:23.671240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:23.671240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:23.671240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.671240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:23.845362 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:23.846187 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:23.846328 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:23.847453 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:23.847802 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:23.847979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.851919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:23.851919 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:23.851919 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:23.851919 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:23.851919 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:23.851919 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:23.851919 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.851919 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:23.851919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.851919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:23.851919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:23.851919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:23.851919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.851919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:26:23.851919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:23.853281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:23.853281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:23.853281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:23.853281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:23.853281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:23.853281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:23.853281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:23.853281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:23.853281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:23.853281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:23.853281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:23.853281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:23.853281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:23.853281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:23.853915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:23.853915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:23.853915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:23.853915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:23.853915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.853915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:23.853915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.853915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:23.853915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:23.853915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:23.853915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:23.853915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:23.853915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.853915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:23.854759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:23.854759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.854759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:23.854759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:23.854759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.854759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:23.854759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:23.854759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.854759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:26:23.854759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:23.854759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:23.854759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:23.854759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:23.854759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:23.854759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:23.854759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:23.854759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:23.854759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:23.854759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:23.854759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:23.855752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:23.855752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:23.855752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:23.855752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:23.855752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:23.855752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:23.855752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:23.855752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:23.855752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.855752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:23.901337 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:23.902053 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:23.902235 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:23.902383 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:26:23.903335 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:23.903703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.904708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:23.904708 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:23.904708 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:23.904708 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:23.904708 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:23.904708 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:23.904708 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.904708 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:23.906175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.906175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:26:23.906175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:23.906175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:23.906175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:23.906175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:23.906175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:23.906175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:23.906175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:23.906175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:23.906175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:23.906175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:26:23.906175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:23.906175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:26:23.906175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:26:23.907163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:23.907163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:26:23.907163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:23.907163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:26:23.907163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:26:23.907163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:23.907163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:26:23.907163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.907163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:26:23.907549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.907549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:23.907549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:23.907549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:23.907549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.907549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:26:23.907549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:26:23.907549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:26:23.907549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:23.907549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:23.907549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:23.907549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:23.908432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:23.908432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:23.908432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:23.908432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:23.908432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:23.908432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:23.908432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:23.908432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:23.908432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:23.908432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:23.908432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:23.908432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:23.908432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:23.908432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:23.908824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:23.908824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:23.908824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.908824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:23.909189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.909189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:23.909189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:23.909189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:23.909189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:23.909189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:23.909189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.909189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:23.909189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:23.909189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.909189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:23.909189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:23.909189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.909189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:23.909189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:23.909189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.910431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:26:23.910431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:26:23.910431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:26:23.910431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:23.910431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:23.910431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:23.910431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:23.910431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:23.910431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:23.910431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:23.910431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:23.910431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:23.910431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:23.910431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:23.910431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:23.910431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:23.910431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:23.910431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:23.910974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:23.910974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:23.910974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:23.910974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:23.910974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:23.910974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.910974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:23.919879 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:23.920779 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:23.920961 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:23.921505 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:23.922217 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:23.922612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.923583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:23.923583 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:23.923583 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:23.923583 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:23.923583 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:23.923583 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:23.923583 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.923583 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:23.924984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.924984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:26:23.924984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:23.924984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:23.924984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:23.924984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:23.924984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:23.924984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:23.924984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:23.924984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:23.924984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:23.924984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:26:23.924984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:23.924984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:26:23.924984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:26:23.925738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:23.925738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:26:23.925738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:23.925738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:26:23.925738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:26:23.925738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:23.925738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:26:23.925738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.925738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:26:23.926422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.926422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:23.926422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:23.926422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:23.926422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.926422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:26:23.926422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:26:23.926422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:23.926422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:23.926422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:23.926422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:23.927148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:23.927148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:23.927148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:23.927148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:23.927148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:23.927148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:23.927148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:23.927148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:23.927148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:23.927148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:23.927148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:23.927148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:23.927148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:23.927148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:23.927927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:23.927927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:23.927927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.927927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:23.928192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.928192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:23.928192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:23.928192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:23.928192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:23.928192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:23.928192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.928192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:23.928192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:23.928192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.928192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:23.928192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:23.928192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.928192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:23.928192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:23.928192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.928192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:26:23.928621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:26:23.928621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:23.928621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:23.928621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:23.928621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:23.928621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:23.928621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:23.928621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:23.928621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:23.928621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:23.928621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:23.928621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:23.928621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:23.928621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:23.928621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:23.928621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:23.928621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:23.928621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:23.928621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:23.929121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:23.929121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:23.929121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.929121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:23.929121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:23.929121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:23.929121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:23.929323 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:23.929854 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:23.930311 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:23.930568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.931189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:23.931189 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:23.931189 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:23.931189 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:23.931189 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:23.931189 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:23.931189 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.931189 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:23.932125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.932125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:23.932125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:23.932125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:23.932125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.932125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:26:23.932125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:23.932125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:23.932125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:23.932125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:23.932125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:23.932125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:23.932504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:23.932504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:23.932504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:23.932504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:23.932504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:23.932504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:23.932504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:23.932504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:23.932504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:23.932504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:23.932504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:23.932504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:23.932504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:23.932891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.932891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:23.933177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.933177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:23.933177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:23.933177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:23.933177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:23.933177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:23.933177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.933177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:23.933177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:23.933177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.933177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:23.933177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:23.933177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.933177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:23.933177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:23.933177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:23.933819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:26:23.933819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:23.933819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:23.933819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:23.933819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:23.933819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:23.933819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:23.933819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:23.933819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:23.933819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:23.933819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:23.933819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:23.933819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:23.933819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:23.933819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:23.933819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:23.933819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:23.933819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:23.933819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:23.934393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:23.934393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:23.934393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:28.612835 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:28.613404 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:28.613521 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:28.614891 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:28.648989 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:28.649442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.650519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:28.650519 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:28.650519 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:28.650519 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:28.650519 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:28.650519 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:28.650519 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.650519 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:28.651257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.651257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:28.651257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:28.651257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:28.651257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.651257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:26:28.651257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:28.651257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:28.651257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:28.651257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:28.651257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:28.651257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:28.652084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:28.652084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:28.652084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:28.652084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:28.652084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:28.652084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:28.652084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:28.652084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:28.652084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:28.652084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:28.652084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:28.652084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:28.652084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:28.653140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.653140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:28.653140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.653140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:28.653140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:28.653140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:28.653140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:28.653140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:28.653140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.653140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:28.653140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:28.653140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.653140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:28.653140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:28.653140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.654139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:28.654139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:28.654139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.654139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:26:28.654139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:28.654139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:28.654139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:28.654139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:28.654139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:28.654139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:28.654139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:28.654139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:28.654139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:28.654139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:28.654139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:28.654139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:28.654139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:28.654139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:28.654139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:28.655151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:28.655151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:28.655151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:28.655151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:28.655151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.655151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:28.847089 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:28.847581 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:28.847849 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:28.848570 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:28.848740 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:28.848953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.849511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:28.849511 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:28.849511 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:28.849511 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:28.849511 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:28.849511 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:28.849511 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.849511 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:28.850477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.850477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:28.850477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:28.850477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:28.850477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.850477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:26:28.850477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:28.850477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:28.850477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:28.850477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:28.850477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:28.850477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:28.851038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:28.851038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:28.851038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:28.851038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:28.851038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:28.851038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:28.851038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:28.851038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:28.851038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:28.851038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:28.851038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:28.851038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:28.851038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:28.851430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.851430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:28.851506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.851506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:28.851506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:28.851506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:28.851506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:28.851506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:28.851506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.851506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:28.851506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:28.851506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.851506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:28.851506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:28.851506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.851506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:28.851506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:28.851506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.852127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:26:28.852127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:28.852127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:28.852127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:28.852127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:28.852127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:28.852127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:28.852127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:28.852127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:28.852127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:28.852127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:28.852127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:28.852127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:28.852127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:28.852127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:28.852127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:28.852127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:28.852127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:28.852127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:28.852843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:28.852843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.852843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:28.902373 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:28.902903 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:28.903121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:28.903319 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:26:28.904114 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:28.904541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.905421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:28.905421 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:28.905421 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:28.905421 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:28.905421 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:28.905421 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:28.905421 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.905421 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:28.906588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.906588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:26:28.906588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:28.906588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:28.906588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:28.906588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:28.906588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:28.906588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:28.906588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:28.906588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:28.906588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:28.906588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:26:28.906588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:28.906588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:26:28.906588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:26:28.907443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:28.907443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:26:28.907443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:28.907443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:26:28.907443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:26:28.907443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:28.907443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:26:28.907443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.907443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:26:28.908027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.908027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:28.908027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:28.908027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:28.908027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.908027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:26:28.908027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:26:28.908027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:26:28.908027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:28.908027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:28.908027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:28.908027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:28.908776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:28.908776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:28.908776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:28.908776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:28.908776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:28.908776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:28.908776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:28.908776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:28.908776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:28.908776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:28.908776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:28.908776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:28.908776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:28.908776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:28.909168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:28.909168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:28.909168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.909168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:28.909613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.909613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:28.909613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:28.909613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:28.909613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:28.909613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:28.909613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.909613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:28.909613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:28.909613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.909613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:28.909613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:28.909613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.909613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:28.909613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:28.909613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.910122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:26:28.910122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:26:28.910122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:26:28.910122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:28.910122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:28.910122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:28.910122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:28.910122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:28.910122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:28.910122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:28.910122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:28.910122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:28.910122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:28.910122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:28.910122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:28.910122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:28.910122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:28.910122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:28.910646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:28.910646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:28.910646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:28.910646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:28.910646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:28.910646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.910646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:28.920628 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:28.920936 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:28.921057 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:28.921203 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:28.921449 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:28.921621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.922065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:28.922065 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:28.922065 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:28.922065 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:28.922065 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:28.922065 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:28.922065 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.922065 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:28.923222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.923222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:26:28.923222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:28.923222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:28.923222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:28.923222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:28.923222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:28.923222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:28.923222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:28.923222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:28.923222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:28.923222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:26:28.923222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:28.923222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:26:28.923222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:26:28.923657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:28.923657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:26:28.923657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:28.923657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:26:28.923657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:26:28.923657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:28.923657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:26:28.923657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.923657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:26:28.924693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.924693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:28.924693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:28.924693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:28.924693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.924693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:26:28.924693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:26:28.924693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:28.924693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:28.924693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:28.924693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:28.925098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:28.925098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:28.925098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:28.925098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:28.925098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:28.925098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:28.925098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:28.925098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:28.925098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:28.925098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:28.925098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:28.925098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:28.925098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:28.925098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:28.925479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:28.925479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:28.925479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.925479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:28.926547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.926547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:28.926547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:28.926547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:28.926547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:28.926547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:28.926547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.926547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:28.926547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:28.926547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.926547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:28.926547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:28.926547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.926547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:28.926547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:28.926547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.926547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:26:28.927298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:26:28.927298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:28.927298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:28.927298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:28.927298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:28.927298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:28.927298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:28.927298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:28.927298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:28.927298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:28.927298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:28.927298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:28.927298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:28.927298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:28.927298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:28.927298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:28.927298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:28.927298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:28.927298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:28.928216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:28.928216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:28.928216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.928216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:28.929129 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:28.929488 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:28.929668 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:28.929938 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:28.930164 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:28.930324 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:28.930562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.930901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:28.930901 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:28.930901 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:28.930901 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:28.930901 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:28.930901 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:28.930901 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.930901 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:28.931747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.931747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:28.931747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:28.931747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:28.931747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.931747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:26:28.931747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:28.931747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:28.931747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:28.931747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:28.931747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:28.931747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:28.932353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:28.932353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:28.932353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:28.932353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:28.932353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:28.932353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:28.932353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:28.932353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:28.932353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:28.932353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:28.932353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:28.932353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:28.932353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:28.932953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.932953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:28.932953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.932953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:28.932953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:28.932953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:28.932953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:28.932953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:28.932953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.932953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:28.932953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:28.932953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.932953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:28.932953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:28.932953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.933694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:28.933694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:28.933694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:28.933694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:26:28.933694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:28.933694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:28.933694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:28.933694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:28.933694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:28.933694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:28.933694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:28.933694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:28.933694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:28.933694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:28.933694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:28.933694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:28.933694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:28.933694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:28.933694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:28.934563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:28.934563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:28.934563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:28.934563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:28.934563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:28.934563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:33.613518 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:33.614366 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:33.614366 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:33.616002 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:33.654538 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:33.654714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.655322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:33.655322 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:33.655322 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:33.655322 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:33.655322 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:33.655322 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:33.655322 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.655322 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:33.656133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.656133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:33.656133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:33.656133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:33.656133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.656133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:26:33.656133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:33.656133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:33.656133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:33.656133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:33.656133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:33.656133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:33.656781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:33.656781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:33.656781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:33.656781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:33.656781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:33.656781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:33.656781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:33.656781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:33.656781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:33.656781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:33.656781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:33.656781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:33.656781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:33.657385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.657385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:33.657631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.657631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:33.657631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:33.657631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:33.657631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:33.657631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:33.657631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.657631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:33.657631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:33.657631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.657631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:33.657631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:33.657631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.657631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:33.657631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:33.657631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.658496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:26:33.658496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:33.658496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:33.658496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:33.658496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:33.658496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:33.658496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:33.658496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:33.658496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:33.658496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:33.658496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:33.658496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:33.658496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:33.658496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:33.658496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:33.658496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:33.658496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:33.658496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:33.658496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:33.659512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:33.659512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.659512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:33.847793 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:33.848270 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:33.848351 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:33.849069 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:33.849203 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:33.849364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.849807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:33.849807 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:33.849807 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:33.849807 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:33.849807 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:33.849807 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:33.849807 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.849807 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:33.850749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.850749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:33.850749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:33.850749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:33.850749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.850749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:26:33.850749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:33.850749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:33.850749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:33.850749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:33.850749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:33.850749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:33.851371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:33.851371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:33.851371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:33.851371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:33.851371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:33.851371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:33.851371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:33.851371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:33.851371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:33.851371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:33.851371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:33.851371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:33.851371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:33.851997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.851997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:33.852148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.852148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:33.852148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:33.852148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:33.852148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:33.852148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:33.852148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.852148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:33.852148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:33.852148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.852148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:33.852148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:33.852148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.852148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:33.852148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:33.852148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.852875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:26:33.852875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:33.852875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:33.852875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:33.852875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:33.852875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:33.852875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:33.852875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:33.852875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:33.852875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:33.852875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:33.852875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:33.852875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:33.852875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:33.852875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:33.852875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:33.852875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:33.852875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:33.852875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:33.853981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:33.853981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.853981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:33.902798 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:33.903358 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:33.903520 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:33.903658 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:26:33.904139 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:33.904414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.905174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:33.905174 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:33.905174 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:33.905174 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:33.905174 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:33.905174 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:33.905174 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.905174 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:33.906075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.906075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:26:33.906075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:33.906075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:33.906075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:33.906075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:33.906075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:33.906075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:33.906075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:33.906075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:33.906075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:33.906075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:26:33.906075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:33.906075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:26:33.906075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:26:33.906800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:33.906800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:26:33.906800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:33.906800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:26:33.906800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:26:33.906800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:33.906800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:26:33.906800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.906800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:26:33.907232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.907232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:33.907232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:33.907232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:33.907232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.907232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:26:33.907232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:26:33.907232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:26:33.907232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:33.907232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:33.907232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:33.907232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:33.907810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:33.907810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:33.907810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:33.907810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:33.907810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:33.907810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:33.907810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:33.907810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:33.907810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:33.907810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:33.907810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:33.907810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:33.907810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:33.907810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:33.908455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:33.908455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:33.908455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.908455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:33.908455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.908455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:33.908455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:33.908455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:33.908455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:33.908455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:33.908455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.908455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:33.908455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:33.908455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.909127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:33.909127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:33.909127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.909127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:33.909127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:33.909127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.909127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:26:33.909127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:26:33.909127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:26:33.909127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:33.909127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:33.909127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:33.909127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:33.909127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:33.909127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:33.909127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:33.909127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:33.909127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:33.909127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:33.910037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:33.910037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:33.910037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:33.910037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:33.910037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:33.910037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:33.910037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:33.910037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:33.910037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:33.910037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:33.910037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.910037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:33.921059 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:33.921457 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:33.921630 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:33.921886 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:33.922290 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:33.922640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.923418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:33.923418 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:33.923418 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:33.923418 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:33.923418 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:33.923418 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:33.923418 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.923418 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:33.924899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.924899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:26:33.924899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:33.924899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:33.924899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:33.924899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:33.924899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:33.924899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:33.924899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:33.924899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:33.924899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:33.924899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:26:33.924899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:33.924899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:26:33.924899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:26:33.925708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:33.925708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:26:33.925708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:33.925708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:26:33.925708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:26:33.925708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:33.925708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:26:33.925708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.925708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:26:33.926401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.926401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:33.926401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:33.926401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:33.926401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.926401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:26:33.926401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:26:33.926401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:33.926401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:33.926401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:33.926401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:33.927006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:33.927006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:33.927006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:33.927006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:33.927006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:33.927006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:33.927006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:33.927006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:33.927006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:33.927006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:33.927006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:33.927006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:33.927006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:33.927006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:33.927628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:33.927628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:33.927628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.927628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:33.928222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.928222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:33.928222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:33.928222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:33.928222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:33.928222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:33.928222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.928222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:33.928222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:33.928222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.928222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:33.928222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:33.928222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.928222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:33.928222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:33.928222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.928222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:26:33.929230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:26:33.929230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:33.929230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:33.929230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:33.929230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:33.929230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:33.929230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:33.929230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:33.929230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:33.929230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:33.929230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:33.929230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:33.929230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:33.929230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:33.929230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:33.929230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:33.929230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:33.929230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:33.929230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:33.930148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:33.930148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:33.930148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.930148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:33.930148 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:33.930148 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:33.930148 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:33.930148 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:33.930609 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:33.930609 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:33.930609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.930796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:33.930796 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:33.930796 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:33.930796 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:33.930796 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:33.930796 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:33.930796 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.930796 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:33.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:33.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:33.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:33.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:26:33.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:33.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:33.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:33.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:33.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:33.931336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:33.931934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:33.931934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:33.931934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:33.931934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:33.931934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:33.931934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:33.931934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:33.931934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:33.931934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:33.931934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:33.931934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:33.931934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:33.931934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:33.932266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.932266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:33.932266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.932266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:33.932266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:33.932266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:33.932266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:33.932266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:33.932266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.932266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:33.932266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:33.932266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.932266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:33.932266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:33.932266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.932641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:33.932641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:33.932641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:33.932641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:26:33.932641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:33.932641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:33.932641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:33.932641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:33.932641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:33.932641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:33.932641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:33.932641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:33.932641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:33.932641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:33.932641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:33.932641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:33.932641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:33.932641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:33.932641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:33.933457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:33.933457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:33.933457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:33.933457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:33.933457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:33.933457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:38.614071 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:38.615161 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:38.615161 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:38.617060 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:38.649852 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:38.650054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.650807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:38.650807 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:38.650807 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:38.650807 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:38.650807 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:38.650807 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:38.650807 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.650807 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:38.651953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.651953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:38.651953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:38.651953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:38.651953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.651953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:26:38.651953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:38.651953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:38.651953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:38.651953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:38.651953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:38.651953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:38.652654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:38.652654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:38.652654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:38.652654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:38.652654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:38.652654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:38.652654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:38.652654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:38.652654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:38.652654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:38.652654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:38.652654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:38.652654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:38.653541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.653541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:38.653541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.653541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:38.653541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:38.653541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:38.653541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:38.653541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:38.653541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.653541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:38.653541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:38.653541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.653541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:38.653541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:38.653541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.654276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:38.654276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:38.654276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.654276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:26:38.654276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:38.654276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:38.654276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:38.654276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:38.654276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:38.654276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:38.654276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:38.654276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:38.654276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:38.654276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:38.654276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:38.654276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:38.654276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:38.654276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:38.654276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:38.655171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:38.655171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:38.655171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:38.655171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:38.655171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.655171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:38.848349 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:38.848847 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:38.849011 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:38.849730 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:38.849952 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:38.850073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.850897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:38.850897 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:38.850897 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:38.850897 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:38.850897 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:38.850897 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:38.850897 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.850897 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:38.852067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.852067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:38.852067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:38.852067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:38.852067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.852067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:26:38.852067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:38.852067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:38.852067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:38.852067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:38.852067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:38.852067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:38.852599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:38.852599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:38.852599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:38.852599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:38.852599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:38.852599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:38.852599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:38.852599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:38.852599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:38.852599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:38.852599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:38.852599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:38.852599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:38.853456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.853456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:38.853456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.853456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:38.853456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:38.853456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:38.853456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:38.853456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:38.853456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.853456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:38.853456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:38.853456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.853456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:38.853456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:38.853456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.853912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:38.853912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:38.853912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.853912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:26:38.853912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:38.853912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:38.853912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:38.853912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:38.853912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:38.853912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:38.853912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:38.853912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:38.853912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:38.853912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:38.853912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:38.853912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:38.853912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:38.853912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:38.853912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:38.854600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:38.854600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:38.854600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:38.854600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:38.854600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.854600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:38.903416 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:38.903998 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:38.904195 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:38.904394 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:26:38.904879 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:38.905095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.905784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:38.905784 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:38.905784 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:38.905784 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:38.905784 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:38.905784 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:38.905784 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.905784 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:38.907097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.907097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:26:38.907097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:38.907097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:38.907097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:38.907097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:38.907097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:38.907097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:38.907097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:38.907097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:38.907097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:38.907097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:26:38.907097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:38.907097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:26:38.907097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:26:38.907932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:38.907932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:26:38.907932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:38.907932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:26:38.907932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:26:38.907932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:38.907932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:26:38.907932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.907932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:26:38.908559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.908559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:38.908559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:38.908559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:38.908559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.908559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:26:38.908559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:26:38.908559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:26:38.908559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:38.908559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:38.908559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:38.908559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:38.909143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:38.909143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:38.909143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:38.909143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:38.909143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:38.909143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:38.909143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:38.909143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:38.909143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:38.909143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:38.909143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:38.909143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:38.909143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:38.909143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:38.909766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:38.909766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:38.909766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.909766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:38.910292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.910292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:38.910292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:38.910292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:38.910292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:38.910292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:38.910292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.910292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:38.910292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:38.910292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.910292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:38.910292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:38.910292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.910292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:38.910292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:38.910292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.911346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:26:38.911346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:26:38.911346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:26:38.911346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:38.911346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:38.911346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:38.911346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:38.911346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:38.911346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:38.911346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:38.911346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:38.911346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:38.911346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:38.911346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:38.911346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:38.911346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:38.911346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:38.911346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:38.912195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:38.912195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:38.912195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:38.912195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:38.912195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:38.912195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.912195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:38.921609 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:38.921982 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:38.922125 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:38.922322 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:38.922931 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:38.923315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.924176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:38.924176 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:38.924176 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:38.924176 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:38.924176 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:38.924176 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:38.924176 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.924176 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:38.925359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.925359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:26:38.925359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:38.925359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:38.925359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:38.925359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:38.925359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:38.925359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:38.925359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:38.925359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:38.925359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:38.925359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:26:38.925359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:38.925359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:26:38.925359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:26:38.926136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:38.926136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:26:38.926136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:38.926136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:26:38.926136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:26:38.926136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:38.926136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:26:38.926136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.926136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:26:38.926827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.926827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:38.926827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:38.926827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:38.926827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.926827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:26:38.926827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:26:38.926827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:38.926827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:38.926827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:38.926827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:38.927426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:38.927426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:38.927426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:38.927426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:38.927426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:38.927426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:38.927426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:38.927426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:38.927426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:38.927426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:38.927426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:38.927426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:38.927426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:38.927426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:38.928256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:38.928256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:38.928256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.928256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:38.928609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.928609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:38.928609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:38.928609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:38.928609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:38.928609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:38.928609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.928609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:38.928609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:38.928609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.928609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:38.928609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:38.928609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.928609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:38.928609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:38.928609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.928609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:26:38.929427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:26:38.929427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:38.929427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:38.929427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:38.929427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:38.929427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:38.929427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:38.929427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:38.929427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:38.929427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:38.929427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:38.929427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:38.929427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:38.929427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:38.929427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:38.929427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:38.929427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:38.929427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:38.929427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:38.930230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:38.930230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:38.930230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.930230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:38.930230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:38.930633 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:38.930633 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:38.930910 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:38.931409 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:38.931643 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:38.931779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.932261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:38.932261 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:38.932261 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:38.932261 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:38.932261 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:38.932261 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:38.932261 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.932261 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:38.933074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.933074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:38.933074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:38.933074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:38.933074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.933074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:26:38.933074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:38.933074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:38.933074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:38.933074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:38.933074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:38.933074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:38.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:38.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:38.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:38.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:38.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:38.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:38.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:38.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:38.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:38.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:38.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:38.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:38.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:38.933797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.933797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:38.934015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.934015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:38.934015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:38.934015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:38.934015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:38.934015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:38.934015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.934015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:38.934015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:38.934015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.934015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:38.934015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:38.934015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.934015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:38.934015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:38.934015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:38.934500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:26:38.934500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:38.934500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:38.934500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:38.934500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:38.934500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:38.934500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:38.934500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:38.934500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:38.934500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:38.934500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:38.934500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:38.934500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:38.934500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:38.934500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:38.934500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:38.934500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:38.934500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:38.934500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:38.935134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:38.935134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:38.935134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:43.614704 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:43.615553 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:43.615553 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:43.616743 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:43.655246 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:43.655625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.656493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:43.656493 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:43.656493 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:43.656493 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:43.656493 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:43.656493 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:43.656493 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.656493 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:43.657487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.657487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:43.657487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:43.657487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:43.657487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.657487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:26:43.657487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:43.657487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:43.657487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:43.657487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:43.657487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:43.657487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:43.657907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:43.657907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:43.657907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:43.657907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:43.657907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:43.657907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:43.657907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:43.657907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:43.657907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:43.657907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:43.657907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:43.657907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:43.657907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:43.658241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.658241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:43.658575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.658575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:43.658575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:43.658575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:43.658575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:43.658575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:43.658575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.658575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:43.658575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:43.658575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.658575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:43.658575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:43.658575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.658575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:43.658575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:43.658575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.659042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:26:43.659042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:43.659042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:43.659042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:43.659042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:43.659042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:43.659042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:43.659042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:43.659042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:43.659042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:43.659042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:43.659042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:43.659042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:43.659042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:43.659042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:43.659042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:43.659042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:43.659042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:43.659042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:43.659531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:43.659531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.659531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:43.849278 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:43.849530 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:43.849797 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:43.853778 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:43.853778 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:43.853894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.854699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:43.854699 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:43.854699 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:43.854699 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:43.854699 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:43.854699 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:43.854699 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.854699 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:43.856144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.856144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:43.856144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:43.856144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:43.856144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.856144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:26:43.856144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:43.856144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:43.856144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:43.856144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:43.856144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:43.856144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:43.856558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:43.856558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:43.856558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:43.856558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:43.856558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:43.856558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:43.856558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:43.856558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:43.856558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:43.856558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:43.856558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:43.856558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:43.856558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:43.856887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.856887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:43.857798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.857798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:43.857798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:43.857798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:43.857798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:43.857798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:43.857798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.857798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:43.857798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:43.857798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.857798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:43.857798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:43.857798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.857798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:43.857798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:43.857798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.859015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:26:43.859015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:43.859015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:43.859015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:43.859015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:43.859015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:43.859015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:43.859015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:43.859015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:43.859015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:43.859015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:43.859015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:43.859015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:43.859015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:43.859015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:43.859015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:43.859015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:43.859015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:43.859015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:43.859795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:43.859795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.859795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:43.903976 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:43.904350 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:43.904525 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:43.904686 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:26:43.905937 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:43.906340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.907305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:43.907305 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:43.907305 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:43.907305 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:43.907305 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:43.907305 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:43.907305 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.907305 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:43.908507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.908507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:26:43.908507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:43.908507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:43.908507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:43.908507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:43.908507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:43.908507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:43.908507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:43.908507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:43.908507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:43.908507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:26:43.908507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:43.908507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:26:43.908507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:26:43.909287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:43.909287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:26:43.909287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:43.909287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:26:43.909287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:26:43.909287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:43.909287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:26:43.909287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.909287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:26:43.909820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.909820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:43.909820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:43.909820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:43.909820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.909820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:26:43.909820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:26:43.909820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:26:43.909820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:43.909820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:43.909820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:43.909820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:43.910424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:43.910424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:43.910424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:43.910424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:43.910424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:43.910424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:43.910424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:43.910424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:43.910424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:43.910424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:43.910424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:43.910424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:43.910424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:43.910424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:43.911053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:43.911053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:43.911053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.911053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:43.911885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.911885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:43.911885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:43.911885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:43.911885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:43.911885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:43.911885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.911885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:43.911885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:43.911885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.911885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:43.911885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:43.911885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.911885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:43.911885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:43.911885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.912505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:26:43.912505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:26:43.912505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:26:43.912505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:43.912505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:43.912505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:43.912505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:43.912505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:43.912505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:43.912505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:43.912505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:43.912505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:43.912505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:43.912505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:43.912505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:43.912505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:43.912505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:43.912505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:43.913032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:43.913032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:43.913032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:43.913032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:43.913032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:43.913032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.913032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:43.921983 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:43.922241 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:43.922415 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:43.922599 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:43.923134 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:43.923571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.924504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:43.924504 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:43.924504 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:43.924504 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:43.924504 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:43.924504 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:43.924504 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.924504 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:43.925790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.925790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:26:43.925790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:43.925790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:43.925790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:43.925790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:43.925790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:43.925790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:43.925790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:43.925790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:43.925790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:43.925790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:26:43.925790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:43.925790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:26:43.925790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:26:43.926572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:43.926572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:26:43.926572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:43.926572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:26:43.926572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:26:43.926572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:43.926572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:26:43.926572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.926572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:26:43.927241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.927241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:43.927241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:43.927241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:43.927241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.927241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:26:43.927241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:26:43.927241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:43.927241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:43.927241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:43.927241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:43.927882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:43.927882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:43.927882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:43.927882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:43.927882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:43.927882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:43.927882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:43.927882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:43.927882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:43.927882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:43.927882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:43.927882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:43.927882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:43.927882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:43.928517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:43.928517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:43.928517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.928517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:43.929118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.929118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:43.929118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:43.929118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:43.929118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:43.929118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:43.929118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.929118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:43.929118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:43.929118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.929118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:43.929118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:43.929118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.929118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:43.929118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:43.929118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.929118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:26:43.929660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:26:43.929660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:43.929660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:43.929660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:43.929660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:43.929660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:43.929660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:43.929660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:43.929660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:43.929660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:43.929660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:43.929660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:43.929660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:43.929660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:43.929660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:43.929660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:43.929660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:43.929660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:43.929660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:43.930184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:43.930184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:43.930184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.930184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:43.930482 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:43.931198 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:43.931320 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:43.931473 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:43.931915 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:43.932288 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:43.932363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.932956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:43.932956 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:43.932956 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:43.932956 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:43.932956 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:43.932956 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:43.932956 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.932956 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:43.934020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.934020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:43.934020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:43.934020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:43.934020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.934020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:26:43.934020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:43.934020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:43.934020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:43.934020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:43.934020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:43.934020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:43.934406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:43.934406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:43.934406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:43.934406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:43.934406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:43.934406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:43.934406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:43.934406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:43.934406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:43.934406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:43.934406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:43.934406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:43.934406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:43.934746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.934746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:43.935412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.935412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:43.935412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:43.935412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:43.935412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:43.935412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:43.935412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.935412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:43.935412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:43.935412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.935412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:43.935412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:43.935412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.935412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:43.935412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:43.935412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:43.936016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:26:43.936016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:43.936016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:43.936016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:43.936016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:43.936016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:43.936016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:43.936016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:43.936016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:43.936016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:43.936016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:43.936016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:43.936016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:43.936016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:43.936016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:43.936016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:43.936016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:43.936016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:43.936016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:43.936505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:43.936505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:43.936505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:48.615338 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:48.616103 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:48.616103 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:48.617041 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:48.658974 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:48.659411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.660217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:48.660217 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:48.660217 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:48.660217 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:48.660217 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:48.660217 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:48.660217 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.660217 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:48.661214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.661214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:48.661214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:48.661214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:48.661214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.661214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:26:48.661214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:48.661214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:48.661214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:48.661214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:48.661214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:48.661214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:48.661828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:48.661828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:48.661828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:48.661828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:48.661828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:48.661828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:48.661828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:48.661828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:48.661828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:48.661828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:48.661828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:48.661828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:48.661828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:48.662367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.662367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:48.662367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.662367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:48.662367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:48.662367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:48.662367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:48.662367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:48.662367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.662367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:48.662367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:48.662367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.662367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:48.662367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:48.662367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.663138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:48.663138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:48.663138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.663138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:26:48.663138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:48.663138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:48.663138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:48.663138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:48.663138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:48.663138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:48.663138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:48.663138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:48.663138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:48.663138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:48.663138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:48.663138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:48.663138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:48.663138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:48.663138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:48.663963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:48.663963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:48.663963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:48.663963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:48.663963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.663963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:48.849634 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:48.850226 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:48.850618 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:48.851732 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:48.851931 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:48.852025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.853591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:48.853591 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:48.853591 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:48.853591 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:48.853591 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:48.853591 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:48.853591 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.853591 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:48.855305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.855305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:48.855305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:48.855305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:48.855305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.855305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:26:48.855305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:48.855305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:48.855305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:48.855305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:48.855305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:48.855305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:48.855991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:48.855991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:48.855991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:48.855991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:48.855991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:48.855991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:48.855991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:48.855991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:48.855991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:48.855991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:48.855991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:48.855991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:48.855991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:48.856377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.856377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:48.857089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.857089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:48.857089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:48.857089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:48.857089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:48.857089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:48.857089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.857089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:48.857089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:48.857089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.857089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:48.857089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:48.857089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.857089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:48.857089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:48.857089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.857612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:26:48.857612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:48.857612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:48.857612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:48.857612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:48.857612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:48.857612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:48.857612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:48.857612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:48.857612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:48.857612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:48.857612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:48.857612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:48.857612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:48.857612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:48.857612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:48.857612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:48.857612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:48.857612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:48.858291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:48.858291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.858291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:48.904451 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:48.904902 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:48.905088 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:48.905268 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:26:48.905891 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:48.908605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.908605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:48.908605 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:48.908605 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:48.908605 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:48.908605 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:48.908605 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:48.908605 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.908605 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:48.908605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.908605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:26:48.908605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:48.908605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:48.908605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:48.908605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:48.908605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:48.908605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:48.909577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:48.909577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:48.909577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:48.909577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:26:48.909577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:48.909577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:26:48.909577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:26:48.909577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:48.909577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:26:48.909577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:48.909577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:26:48.909577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:26:48.909577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:48.909577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:26:48.909577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.909577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:26:48.910407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.910407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:48.910407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:48.910407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:48.910407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.910407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:26:48.910407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:26:48.910407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:26:48.910407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:48.910407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:48.910407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:48.910407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:48.911139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:48.911139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:48.911139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:48.911139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:48.911139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:48.911139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:48.911139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:48.911139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:48.911139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:48.911139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:48.911139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:48.911139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:48.911139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:48.911139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:48.911816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:48.911816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:48.911816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.911816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:48.911816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.911816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:48.911816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:48.911816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:48.911816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:48.911816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:48.911816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.911816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:48.911816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:48.911816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.912454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:48.912454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:48.912454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.912454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:48.912454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:48.912454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.912454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:26:48.912454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:26:48.912454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:26:48.912454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:48.912454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:48.912454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:48.912454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:48.912454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:48.912454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:48.912454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:48.912454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:48.912454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:48.912454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:48.913433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:48.913433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:48.913433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:48.913433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:48.913433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:48.913433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:48.913433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:48.913433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:48.913433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:48.913433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:48.913433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.913433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:48.922505 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:48.922918 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:48.923151 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:48.923370 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:48.923939 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:48.924341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.925149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:48.925149 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:48.925149 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:48.925149 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:48.925149 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:48.925149 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:48.925149 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.925149 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:48.926307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.926307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:26:48.926307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:48.926307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:48.926307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:48.926307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:48.926307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:48.926307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:48.926307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:48.926307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:48.926307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:48.926307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:26:48.926307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:48.926307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:26:48.926307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:26:48.927176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:48.927176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:26:48.927176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:48.927176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:26:48.927176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:26:48.927176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:48.927176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:26:48.927176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.927176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:26:48.927753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.927753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:48.927753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:48.927753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:48.927753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.927753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:26:48.927753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:26:48.927753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:48.927753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:48.927753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:48.927753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:48.928265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:48.928265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:48.928265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:48.928265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:48.928265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:48.928265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:48.928265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:48.928265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:48.928265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:48.928265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:48.928265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:48.928265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:48.928265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:48.928265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:48.928878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:48.928878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:48.928878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.928878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:48.929652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.929652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:48.929652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:48.929652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:48.929652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:48.929652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:48.929652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.929652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:48.929652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:48.929652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.929652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:48.929652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:48.929652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.929652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:48.929652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:48.929652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.929652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:26:48.930435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:26:48.930435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:48.930435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:48.930435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:48.930435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:48.930435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:48.930435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:48.930435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:48.930435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:48.930435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:48.930435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:48.930435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:48.930435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:48.930435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:48.930435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:48.930435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:48.930435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:48.930435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:48.930435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:48.931310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:48.931310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:48.931310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.931310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:48.931310 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:48.931495 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:48.931602 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:48.931783 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:48.932135 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:48.932383 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:48.932556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.933278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:48.933278 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:48.933278 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:48.933278 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:48.933278 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:48.933278 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:48.933278 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.933278 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:48.934103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.934103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:48.934103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:48.934103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:48.934103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.934103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:26:48.934103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:48.934103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:48.934103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:48.934103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:48.934103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:48.934103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:48.934485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:48.934485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:48.934485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:48.934485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:48.934485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:48.934485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:48.934485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:48.934485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:48.934485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:48.934485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:48.934485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:48.934485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:48.934485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:48.934873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.934873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:48.935213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.935213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:48.935213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:48.935213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:48.935213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:48.935213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:48.935213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.935213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:48.935213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:48.935213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.935213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:48.935213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:48.935213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.935213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:48.935213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:48.935213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:48.935706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:26:48.935706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:48.935706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:48.935706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:48.935706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:48.935706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:48.935706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:48.935706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:48.935706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:48.935706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:48.935706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:48.935706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:48.935706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:48.935706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:48.935706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:48.935706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:48.935706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:48.935706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:48.935706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:48.936225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:48.936225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:48.936225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:53.615981 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:53.616862 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:53.616862 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:53.618105 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:53.648404 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:53.648887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.649700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:53.649700 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:53.649700 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:53.649700 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:53.649700 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:53.649700 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:53.649700 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.649700 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:53.650442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.650442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:53.650442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:53.650442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:53.650442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.650442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:26:53.650442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:53.650442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:53.650442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:53.650442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:53.650442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:53.650442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:53.651068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:53.651068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:53.651068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:53.651068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:53.651068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:53.651068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:53.651068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:53.651068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:53.651068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:53.651068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:53.651068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:53.651068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:53.651068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:53.651879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.651879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:53.651879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.651879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:53.651879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:53.651879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:53.651879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:53.651879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:53.651879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.651879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:53.651879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:53.651879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.651879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:53.651879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:53.651879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.652650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:53.652650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:53.652650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.652650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:26:53.652650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:53.652650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:53.652650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:53.652650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:53.652650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:53.652650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:53.652650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:53.652650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:53.652650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:53.652650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:53.652650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:53.652650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:53.652650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:53.652650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:53.652650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:53.653609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:53.653609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:53.653609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:53.653609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:53.653609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.653609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:53.850655 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:53.851091 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:53.851297 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:53.853780 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:53.854413 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:53.856053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.856617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:53.856617 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:53.856617 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:53.856617 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:53.856617 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:53.856617 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:53.856617 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.856617 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:53.857802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.857802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:53.857802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:53.857802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:53.857802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.857802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:26:53.857802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:53.857802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:53.857802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:53.857802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:53.857802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:53.857802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:53.858295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:53.858295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:53.858295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:53.858295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:53.858295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:53.858295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:53.858295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:53.858295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:53.858295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:53.858295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:53.858295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:53.858295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:53.858295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:53.858655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.858655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:53.858908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.858908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:53.858908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:53.858908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:53.858908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:53.858908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:53.858908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.858908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:53.858908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:53.858908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.858908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:53.858908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:53.858908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.858908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:53.858908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:53.858908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.859385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:26:53.859385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:53.859385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:53.859385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:53.859385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:53.859385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:53.859385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:53.859385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:53.859385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:53.859385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:53.859385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:53.859385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:53.859385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:53.859385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:53.859385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:53.859385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:53.859385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:53.859385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:53.859385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:53.859962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:53.859962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.859962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:53.905193 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:53.906081 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:53.906220 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:53.906454 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:26:53.907262 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:53.907614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.908795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:53.908795 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:53.908795 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:53.908795 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:53.908795 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:53.908795 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:53.908795 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.908795 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:53.910573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.910573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:26:53.910573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:53.910573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:53.910573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:53.910573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:53.910573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:53.910573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:53.910573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:53.910573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:53.910573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:53.910573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:26:53.910573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:53.910573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:26:53.910573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:26:53.911304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:53.911304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:26:53.911304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:53.911304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:26:53.911304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:26:53.911304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:53.911304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:26:53.911304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.911304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:26:53.912061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.912061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:53.912061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:53.912061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:53.912061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.912061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:26:53.912061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:26:53.912061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:26:53.912061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:53.912061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:53.912061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:53.912061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:53.912740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:53.912740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:53.912740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:53.912740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:53.912740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:53.912740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:53.912740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:53.912740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:53.912740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:53.912740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:53.912740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:53.912740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:53.912740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:53.912740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:53.913372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:53.913372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:53.913372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.913372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:53.913923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.913923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:53.913923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:53.913923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:53.913923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:53.913923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:53.913923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.913923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:53.913923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:53.913923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.913923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:53.913923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:53.913923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.913923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:53.913923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:53.913923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.914603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:26:53.914603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:26:53.914603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:26:53.914603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:53.914603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:53.914603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:53.914603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:53.914603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:53.914603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:53.914603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:53.914603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:53.914603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:53.914603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:53.914603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:53.914603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:53.914603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:53.914603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:53.914603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:53.915167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:53.915167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:53.915167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:53.915167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:53.915167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:53.915167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.915167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:53.923049 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:53.923465 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:53.923651 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:53.923896 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:53.924385 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:53.924699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.925388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:53.925388 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:53.925388 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:53.925388 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:53.925388 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:53.925388 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:53.925388 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.925388 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:53.926642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.926642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:26:53.926642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:53.926642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:53.926642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:53.926642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:53.926642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:53.926642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:53.926642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:53.926642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:53.926642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:53.926642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:26:53.926642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:53.926642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:26:53.926642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:26:53.927293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:53.927293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:26:53.927293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:53.927293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:26:53.927293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:26:53.927293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:53.927293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:26:53.927293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.927293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:26:53.928171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.928171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:53.928171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:53.928171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:53.928171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.928171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:26:53.928171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:26:53.928171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:53.928171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:53.928171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:53.928171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:53.928726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:53.928726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:53.928726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:53.928726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:53.928726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:53.928726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:53.928726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:53.928726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:53.928726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:53.928726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:53.928726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:53.928726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:53.928726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:53.928726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:53.929308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:53.929308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:53.929308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.929308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:53.930107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.930107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:53.930107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:53.930107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:53.930107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:53.930107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:53.930107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.930107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:53.930107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:53.930107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.930107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:53.930107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:53.930107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.930107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:53.930107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:53.930107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.930107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:26:53.930869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:26:53.930869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:53.930869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:53.930869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:53.930869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:53.930869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:53.930869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:53.930869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:53.930869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:53.930869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:53.930869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:53.930869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:53.930869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:53.930869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:53.930869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:53.930869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:53.930869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:53.930869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:53.930869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:53.931637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:53.931637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:53.931637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.931637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:53.931637 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:53.932100 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:53.932291 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:53.932518 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:53.933119 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:53.933496 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:53.933763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.934511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:53.934511 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:53.934511 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:53.934511 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:53.934511 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:53.934511 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:53.934511 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.934511 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:53.935550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.935550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:53.935550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:53.935550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:53.935550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.935550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:26:53.935550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:53.935550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:53.935550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:53.935550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:53.935550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:53.935550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:53.935907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:53.935907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:53.935907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:53.935907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:53.935907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:53.935907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:53.935907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:53.935907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:53.935907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:53.935907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:53.935907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:53.935907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:53.935907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:53.936232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.936232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:53.936571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.936571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:53.936571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:53.936571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:53.936571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:53.936571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:53.936571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.936571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:53.936571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:53.936571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.936571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:53.936571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:53.936571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.936571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:53.936571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:53.936571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:53.937027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:26:53.937027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:53.937027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:53.937027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:53.937027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:53.937027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:53.937027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:53.937027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:53.937027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:53.937027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:53.937027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:53.937027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:53.937027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:53.937027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:53.937027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:53.937027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:53.937027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:53.937027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:53.937027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:53.937508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:53.937508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:53.937508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:58.616612 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:58.617623 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:58.617623 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:58.617939 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:58.654565 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:58.654848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.655841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:58.655841 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:58.655841 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:58.655841 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:58.655841 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:58.655841 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:58.655841 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.655841 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:58.656714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.656714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:58.656714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:58.656714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:58.656714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.656714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:26:58.656714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:58.656714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:58.656714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:58.656714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:58.656714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:58.656714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:58.657886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:58.657886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:58.657886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:58.657886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:58.657886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:58.657886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:58.657886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:58.657886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:58.657886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:58.657886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:58.657886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:58.657886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:58.657886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:58.658616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.658616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:58.658616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.658616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:58.658616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:58.658616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:58.658616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:58.658616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:58.658616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.658616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:58.658616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:58.658616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.658616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:58.658616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:58.658616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.660615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:58.660615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:58.660615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.660615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:26:58.660615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:58.660615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:58.660615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:58.660615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:58.660615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:58.660615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:58.660615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:58.660615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:58.660615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:58.660615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:58.660615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:58.660615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:58.660615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:58.660615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:58.660615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:58.661128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:58.661128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:58.661128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:58.661128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:58.661128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.661128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:58.851160 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:58.851839 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:58.851999 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:58.852989 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:58.853805 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:58.854053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.854992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:58.854992 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:58.854992 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:58.854992 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:58.854992 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:58.854992 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:58.854992 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.854992 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:58.856390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.856390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:58.856390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:58.856390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:58.856390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.856390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:26:58.856390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:58.856390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:58.856390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:58.856390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:58.856390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:58.856390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:58.857136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:58.857136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:58.857136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:58.857136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:58.857136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:58.857136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:58.857136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:58.857136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:58.857136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:58.857136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:58.857136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:58.857136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:58.857136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:58.858518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.858518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:58.858518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.858518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:58.858518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:58.858518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:58.858518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:58.858518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:58.858518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.858518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:58.858518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:58.858518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.858518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:58.858518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:58.858518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.859350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:58.859350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:58.859350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.859350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:26:58.859350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:58.859350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:58.859350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:58.859350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:58.859350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:58.859350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:58.859350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:58.859350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:58.859350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:58.859350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:58.859350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:58.859350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:58.859350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:58.859350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:58.859350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:58.861380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:58.861380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:58.861380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:58.861380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:58.861380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.861380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:58.905956 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:58.906659 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:58.906895 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:58.907101 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:26:58.907874 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:58.908219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.909182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:58.909182 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:58.909182 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:58.909182 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:58.909182 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:58.909182 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:58.909182 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.909182 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:58.910608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.910608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:26:58.910608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:58.910608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:58.910608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:58.910608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:58.910608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:58.910608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:58.910608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:58.910608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:58.910608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:58.910608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:26:58.910608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:58.910608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:26:58.910608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:26:58.911337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:58.911337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:26:58.911337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:58.911337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:26:58.911337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:26:58.911337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:58.911337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:26:58.911337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.911337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:26:58.912100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.912100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:58.912100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:58.912100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:58.912100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.912100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:26:58.912100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:26:58.912100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:26:58.912100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:58.912100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:58.912100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:58.912100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:58.912629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:58.912629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:58.912629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:58.912629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:58.912629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:58.912629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:58.912629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:58.912629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:58.912629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:58.912629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:58.912629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:58.912629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:58.912629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:58.912629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:58.913297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:58.913297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:58.913297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.913297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:58.913882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.913882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:58.913882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:58.913882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:58.913882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:58.913882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:58.913882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.913882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:58.913882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:58.913882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.913882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:58.913882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:58.913882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.913882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:58.913882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:58.913882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.914563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:26:58.914563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:26:58.914563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:26:58.914563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:58.914563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:58.914563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:58.914563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:58.914563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:58.914563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:58.914563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:58.914563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:58.914563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:58.914563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:58.914563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:58.914563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:58.914563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:58.914563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:58.914563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:58.915108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:58.915108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:58.915108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:58.915108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:58.915108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:58.915108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.915108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:58.923588 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:58.923872 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:58.923981 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:58.924131 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:58.924514 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:58.924768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.925400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:58.925400 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:58.925400 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:58.925400 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:58.925400 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:58.925400 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:58.925400 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.925400 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:58.926270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.926270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:26:58.926270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:58.926270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:58.926270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:58.926270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:58.926270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:58.926270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:58.926270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:58.926270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:26:58.926270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:58.926270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:26:58.926270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:58.926270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:26:58.926270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:26:58.927085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:58.927085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:26:58.927085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:58.927085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:26:58.927085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:26:58.927085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:58.927085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:26:58.927085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.927085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:26:58.927530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.927530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:58.927530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:58.927530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:58.927530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.927530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:26:58.927530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:26:58.927530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:58.927530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:58.927530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:58.927530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:58.928046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:58.928046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:58.928046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:58.928046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:58.928046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:58.928046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:58.928046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:58.928046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:58.928046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:58.928046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:58.928046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:58.928046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:58.928046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:58.928046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:58.928703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:58.928703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:58.928703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.928703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:58.929127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.929127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:58.929127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:58.929127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:58.929127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:58.929127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:58.929127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.929127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:58.929127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:58.929127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.929127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:58.929127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:58.929127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.929127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:58.929127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:58.929127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.929127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:26:58.929920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:26:58.929920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:26:58.929920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:26:58.929920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:58.929920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:58.929920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:58.929920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:58.929920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:58.929920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:58.929920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:58.929920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:58.929920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:58.929920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:58.929920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:58.929920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:58.929920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:58.929920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:58.929920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:58.929920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:58.930788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:58.930788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:58.930788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.930788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:26:58.932127 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:26:58.932444 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:26:58.932611 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:26:58.932884 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:26:58.933483 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:58.934040 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:26:58.934381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.935337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:26:58.935337 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:58.935337 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:26:58.935337 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:26:58.935337 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:58.935337 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:26:58.935337 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.935337 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:26:58.936930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.936930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:26:58.936930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:58.936930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:26:58.936930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.936930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:26:58.936930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:58.936930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:58.936930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:58.936930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:58.936930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:58.936930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:58.937300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:26:58.937300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:58.937300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:26:58.937300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:58.937300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:26:58.937300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:26:58.937300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:58.937300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:26:58.937300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:58.937300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:26:58.937300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:26:58.937300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:58.937300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:26:58.937647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.937647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:26:58.938704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.938704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:26:58.938704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:26:58.938704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:26:58.938704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:26:58.938704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:26:58.938704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.938704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:26:58.938704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:26:58.938704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.938704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:26:58.938704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:26:58.938704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.938704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:26:58.938704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:26:58.938704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:26:58.939152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:26:58.939152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:26:58.939152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:58.939152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:58.939152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:26:58.939152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:26:58.939152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:26:58.939152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:26:58.939152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:26:58.939152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:26:58.939152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:26:58.939152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:26:58.939152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:26:58.939152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:26:58.939152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:26:58.939152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:26:58.939152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:26:58.939152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:26:58.939152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:26:58.939744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:26:58.939744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:26:58.939744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:03.617338 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:03.617713 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:03.617877 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:03.619010 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:03.624830 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=434 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:27:03.652383 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:03.652663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.653349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:03.653349 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:03.653349 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:03.653349 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:03.653349 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:03.653349 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:03.653349 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.653349 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:03.654198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.654198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:03.654198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:03.654198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:03.654198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.654198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:27:03.654198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:03.654198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:03.654198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:03.654198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:03.654198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:03.654198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:03.654598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:03.654598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:03.654598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:03.654598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:03.654598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:03.654598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:03.654598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:03.654598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:03.654598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:03.654598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:03.654598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:03.654598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:03.654598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:03.655038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.655038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:03.655260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.655260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:03.655260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:03.655260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:03.655260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:03.655260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:03.655260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.655260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:03.655260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:03.655260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.655260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:03.655260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:03.655260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.655260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:03.655260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:03.655260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.655670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:27:03.655670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:03.655670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:03.655670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:03.655670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:03.655670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:03.655670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:03.655670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:03.655670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:03.655670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:03.655670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:03.655670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:03.655670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:03.655670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:03.655670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:03.655670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:03.655670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:03.655670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:03.655670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:03.656168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:03.656168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.656168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:03.852307 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:03.853336 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:03.853650 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:03.854605 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:03.854798 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:03.854934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.855508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:03.855508 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:03.855508 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:03.855508 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:03.855508 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:03.855508 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:03.855508 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.855508 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:03.856990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.856990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:03.856990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:03.856990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:03.856990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.856990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:27:03.856990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:03.856990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:03.856990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:03.856990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:03.856990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:03.856990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:03.857383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:03.857383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:03.857383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:03.857383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:03.857383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:03.857383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:03.857383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:03.857383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:03.857383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:03.857383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:03.857383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:03.857383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:03.857383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:03.857737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.857737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:03.858609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.858609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:03.858609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:03.858609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:03.858609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:03.858609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:03.858609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.858609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:03.858609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:03.858609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.858609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:03.858609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:03.858609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.858609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:03.858609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:03.858609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.859126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:27:03.859126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:03.859126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:03.859126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:03.859126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:03.859126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:03.859126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:03.859126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:03.859126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:03.859126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:03.859126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:03.859126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:03.859126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:03.859126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:03.859126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:03.859126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:03.859126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:03.859126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:03.859126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:03.859617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:03.859617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.859617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:03.906889 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:03.907253 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:03.907435 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:03.907621 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:27:03.908510 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:03.908851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.909637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:03.909637 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:03.909637 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:03.909637 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:03.909637 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:03.909637 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:03.909637 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.909637 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:03.910938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.910938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:27:03.910938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:03.910938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:03.910938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:03.910938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:03.910938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:03.910938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:03.910938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:03.910938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:03.910938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:03.910938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:27:03.910938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:03.910938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:27:03.910938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:27:03.911662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:03.911662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:27:03.911662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:03.911662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:27:03.911662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:27:03.911662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:03.911662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:27:03.911662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.911662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:27:03.912363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.912363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:03.912363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:03.912363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:03.912363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.912363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:27:03.912363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:27:03.912363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:27:03.912363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:03.912363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:03.912363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:03.912363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:03.912906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:03.912906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:03.912906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:03.912906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:03.912906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:03.912906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:03.912906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:03.912906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:03.912906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:03.912906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:03.912906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:03.912906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:03.912906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:03.912906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:03.913258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:03.913258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:03.913258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.913258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:03.914070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.914070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:03.914070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:03.914070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:03.914070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:03.914070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:03.914070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.914070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:03.914070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:03.914070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.914070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:03.914070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:03.914070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.914070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:03.914070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:03.914070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.914570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:27:03.914570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:27:03.914570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:27:03.914570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:03.914570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:03.914570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:03.914570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:03.914570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:03.914570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:03.914570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:03.914570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:03.914570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:03.914570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:03.914570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:03.914570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:03.914570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:03.914570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:03.914570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:03.915047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:03.915047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:03.915047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:03.915047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:03.915047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:03.915047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.915047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:03.924108 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:03.924425 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:03.924607 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:03.924866 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:03.925320 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:03.925653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.926443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:03.926443 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:03.926443 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:03.926443 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:03.926443 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:03.926443 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:03.926443 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.926443 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:03.927839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.927839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:27:03.927839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:03.927839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:03.927839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:03.927839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:03.927839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:03.927839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:03.927839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:03.927839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:03.927839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:03.927839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:27:03.927839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:03.927839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:27:03.927839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:27:03.928571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:03.928571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:27:03.928571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:03.928571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:27:03.928571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:27:03.928571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:03.928571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:27:03.928571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.928571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:27:03.929320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.929320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:03.929320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:03.929320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:03.929320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.929320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:27:03.929320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:27:03.929320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:03.929320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:03.929320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:03.929320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:03.929910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:03.929910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:03.929910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:03.929910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:03.929910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:03.929910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:03.929910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:03.929910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:03.929910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:03.929910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:03.929910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:03.929910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:03.929910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:03.929910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:03.930524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:03.930524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:03.930524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.930524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:03.931479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.931479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:03.931479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:03.931479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:03.931479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:03.931479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:03.931479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.931479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:03.931479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:03.931479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.931479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:03.931479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:03.931479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.931479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:03.931479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:03.931479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.931479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:27:03.932903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:27:03.932903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:03.932903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:03.932903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:03.932903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:03.932903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:03.932903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:03.932903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:03.932903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:03.932903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:03.932903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:03.932903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:03.932903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:03.932903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:03.932903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:03.932903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:03.932903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:03.932903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:03.932903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:03.933891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:03.933891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:03.933891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.933891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:03.933891 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:03.933891 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:03.933891 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:03.933891 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:03.934392 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:03.934392 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:03.934392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.934392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:03.934392 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:03.934392 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:03.934392 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:03.934392 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:03.934392 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:03.934392 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.934392 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:03.935241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.935241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:03.935241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:03.935241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:03.935241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.935241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:27:03.935241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:03.935241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:03.935241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:03.935241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:03.935241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:03.935241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:03.935908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:03.935908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:03.935908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:03.935908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:03.935908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:03.935908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:03.935908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:03.935908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:03.935908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:03.935908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:03.935908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:03.935908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:03.935908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:03.936347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.936347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:03.936347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.936347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:03.936347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:03.936347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:03.936347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:03.936347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:03.936347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.936347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:03.936347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:03.936347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.936347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:03.936347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:03.936347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.936759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:03.936759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:03.936759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:03.936759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:27:03.936759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:03.936759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:03.936759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:03.936759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:03.936759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:03.936759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:03.936759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:03.936759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:03.936759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:03.936759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:03.936759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:03.936759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:03.936759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:03.936759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:03.936759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:03.937248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:03.937248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:03.937248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:03.937248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:03.937248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:03.937248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:08.617977 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:08.618734 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:08.618734 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:08.619350 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:08.698525 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:08.698923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.699598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:08.699598 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:08.699598 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:08.699598 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:08.699598 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:08.699598 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:08.699598 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.699598 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:08.700460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.081 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.700460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:08.700460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:08.700460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:08.700460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.700460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:27:08.700460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:08.700460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:08.700460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:08.700460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:08.700460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:08.700460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:08.701122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:08.701122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:08.701122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:08.701122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:08.701122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:08.701122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:08.701122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:08.701122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:08.701122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:08.701122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:08.701122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:08.701122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:08.701122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:08.702144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.702144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:08.702144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.702144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:08.702144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:08.702144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:08.702144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:08.702144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:08.702144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.702144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:08.702144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:08.702144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.702144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:08.702144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:08.702144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.702576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:08.702576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:08.702576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.702576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:27:08.702576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:08.702576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:08.702576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:08.702576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:08.702576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:08.702576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:08.702576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:08.702576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:08.702576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:08.702576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:08.702576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:08.702576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:08.702576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:08.702576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:08.702576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:08.703117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:08.703117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:08.703117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:08.703117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:08.703117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.703117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:08.853736 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:08.854413 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:08.854413 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:08.855517 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:08.855862 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:08.856174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.857808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:08.857808 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:08.857808 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:08.857808 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:08.857808 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:08.857808 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:08.857808 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.857808 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:08.858836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.858836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:08.858836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:08.858836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:08.858836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.858836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:27:08.858836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:08.858836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:08.858836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:08.858836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:08.858836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:08.858836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:08.859469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:08.859469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:08.859469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:08.859469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:08.859469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:08.859469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:08.859469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:08.859469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:08.859469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:08.859469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:08.859469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:08.859469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:08.859469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:08.860076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.860076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:08.861088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.861088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:08.861088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:08.861088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:08.861088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:08.861088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:08.861088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.861088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:08.861088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:08.861088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.861088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:08.861088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:08.861088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.861088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:08.861088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:08.861088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.861937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:27:08.861937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:08.861937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:08.861937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:08.861937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:08.861937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:08.861937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:08.861937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:08.861937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:08.861937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:08.861937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:08.861937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:08.861937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:08.861937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:08.861937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:08.861937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:08.861937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:08.861937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:08.861937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:08.862796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:08.862796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.862796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:08.907428 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:08.908034 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:08.908232 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:08.908440 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:27:08.909063 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:08.909457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.910526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:08.910526 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:08.910526 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:08.910526 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:08.910526 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:08.910526 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:08.910526 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.910526 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:08.911836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.911836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:27:08.911836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:08.911836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:08.911836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:08.911836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:08.911836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:08.911836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:08.911836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:08.911836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:08.911836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:08.911836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:27:08.911836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:08.911836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:27:08.911836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:27:08.912533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:08.912533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:27:08.912533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:08.912533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:27:08.912533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:27:08.912533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:08.912533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:27:08.912533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.912533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:27:08.913200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.913200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:08.913200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:08.913200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:08.913200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.913200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:27:08.913200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:27:08.913200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:27:08.913200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:08.913200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:08.913200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:08.913200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:08.913802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:08.913802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:08.913802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:08.913802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:08.913802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:08.913802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:08.913802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:08.913802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:08.913802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:08.913802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:08.913802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:08.913802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:08.913802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:08.913802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:08.914434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:08.914434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:08.914434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.914434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:08.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:08.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:08.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:08.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:08.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:08.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:08.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:08.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:08.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:08.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:08.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:08.914615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.915426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:27:08.915426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:27:08.915426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:27:08.915426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:08.915426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:08.915426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:08.915426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:08.915426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:08.915426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:08.915426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:08.915426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:08.915426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:08.915426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:08.915426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:08.915426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:08.915426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:08.915426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:08.915426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:08.916298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:08.916298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:08.916298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:08.916298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:08.916298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:08.916298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.916298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:08.924485 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:08.924765 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:08.924904 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:08.925053 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:08.925605 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:08.926034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.926812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:08.926812 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:08.926812 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:08.926812 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:08.926812 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:08.926812 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:08.926812 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.926812 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:08.927969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.927969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:27:08.927969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:08.927969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:08.927969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:08.927969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:08.927969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:08.927969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:08.927969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:08.927969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:08.927969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:08.927969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:27:08.927969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:08.927969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:27:08.927969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:27:08.928634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:08.928634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:27:08.928634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:08.928634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:27:08.928634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:27:08.928634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:08.928634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:27:08.928634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.928634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:27:08.929442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.929442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:08.929442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:08.929442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:08.929442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.929442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:27:08.929442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:27:08.929442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:08.929442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:08.929442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:08.929442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:08.929944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:08.929944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:08.929944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:08.929944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:08.929944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:08.929944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:08.929944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:08.929944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:08.929944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:08.929944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:08.929944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:08.929944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:08.929944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:08.929944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:08.930562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:08.930562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:08.930562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.930562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:08.933186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.933186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:08.933186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:08.933186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:08.933186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:08.933186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:08.933186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.933186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:08.933186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:08.933186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.933186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:08.933186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:08.933186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.933186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:08.933186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:08.933186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.933186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:27:08.933839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:27:08.933839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:08.933839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:08.933839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:08.933839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:08.933839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:08.933839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:08.933839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:08.933839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:08.933839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:08.933839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:08.933839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:08.933839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:08.933839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:08.933839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:08.933839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:08.933839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:08.933839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:08.933839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:08.934420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:08.934420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:08.934420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.934420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:08.934420 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:08.934420 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:08.934420 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:08.934420 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:08.934711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:08.934711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:08.934711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.934711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:08.934711 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:08.934711 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:08.934711 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:08.934711 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:08.934711 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:08.934711 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.934711 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:08.935203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.935203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:08.935203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:08.935203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:08.935203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.935203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:27:08.935203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:08.935203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:08.935203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:08.935203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:08.935203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:08.935203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:08.935789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:08.935789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:08.935789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:08.935789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:08.935789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:08.935789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:08.935789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:08.935789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:08.935789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:08.935789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:08.935789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:08.935789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:08.935789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:08.936345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.936345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:08.936345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.936345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:08.936345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:08.936345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:08.936345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:08.936345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:08.936345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.936345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:08.936345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:08.936345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.936345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:08.936345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:08.936345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.937046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:08.937046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:08.937046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:08.937046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:27:08.937046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:08.937046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:08.937046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:08.937046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:08.937046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:08.937046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:08.937046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:08.937046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:08.937046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:08.937046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:08.937046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:08.937046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:08.937046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:08.937046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:08.937046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:08.937897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:08.937897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:08.937897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:08.937897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:08.937897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:08.937897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:13.618625 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:13.619574 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:13.619574 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:13.620486 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:13.656356 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:13.656356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.656594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:13.656594 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:13.656594 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:13.656594 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:13.656594 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:13.656594 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:13.656594 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.656594 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:13.657835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.657835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:13.657835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:13.657835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:13.657835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.657835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:27:13.657835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:13.657835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:13.657835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:13.657835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:13.657835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:13.657835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:13.658646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:13.658646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:13.658646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:13.658646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:13.658646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:13.658646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:13.658646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:13.658646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:13.658646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:13.658646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:13.658646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:13.658646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:13.658646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:13.659337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.659337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:13.659337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.659337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:13.659337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:13.659337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:13.659337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:13.659337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:13.659337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.659337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:13.659337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:13.659337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.659337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:13.659337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:13.659337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.660109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:13.660109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:13.660109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.660109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:27:13.660109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:13.660109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:13.660109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:13.660109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:13.660109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:13.660109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:13.660109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:13.660109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:13.660109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:13.660109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:13.660109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:13.660109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:13.660109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:13.660109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:13.660109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:13.661042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:13.661042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:13.661042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:13.661042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:13.661042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.661042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:13.854087 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:13.854611 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:13.854830 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:13.855831 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:13.856105 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:13.856334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.857142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:13.857142 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:13.857142 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:13.857142 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:13.857142 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:13.857142 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:13.857142 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.857142 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:13.858506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.858506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:13.858506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:13.858506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:13.858506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.858506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:27:13.858506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:13.858506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:13.858506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:13.858506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:13.858506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:13.858506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:13.858925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:13.858925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:13.858925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:13.858925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:13.858925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:13.858925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:13.858925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:13.858925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:13.858925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:13.858925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:13.858925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:13.858925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:13.858925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:13.859412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.859412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:13.860191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.860191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:13.860191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:13.860191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:13.860191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:13.860191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:13.860191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.860191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:13.860191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:13.860191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.860191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:13.860191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:13.860191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.860191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:13.860191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:13.860191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.860707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:27:13.860707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:13.860707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:13.860707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:13.860707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:13.860707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:13.860707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:13.860707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:13.860707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:13.860707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:13.860707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:13.860707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:13.860707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:13.860707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:13.860707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:13.860707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:13.860707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:13.860707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:13.860707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:13.861280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:13.861280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.861280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:13.908217 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:13.908563 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:13.908774 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:13.908958 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:27:13.909414 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:13.909771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.910448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:13.910448 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:13.910448 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:13.910448 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:13.910448 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:13.910448 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:13.910448 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.910448 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:13.911619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.911619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:27:13.911619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:13.911619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:13.911619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:13.911619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:13.911619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:13.911619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:13.911619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:13.911619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:13.911619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:13.911619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:27:13.911619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:13.911619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:27:13.911619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:27:13.912476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:13.912476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:27:13.912476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:13.912476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:27:13.912476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:27:13.912476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:13.912476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:27:13.912476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.912476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:27:13.912919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.912919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:13.912919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:13.912919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:13.912919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.912919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:27:13.912919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:27:13.912919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:27:13.912919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:13.912919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:13.912919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:13.912919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:13.913480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:13.913480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:13.913480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:13.913480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:13.913480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:13.913480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:13.913480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:13.913480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:13.913480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:13.913480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:13.913480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:13.913480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:13.913480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:13.913480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:13.914122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:13.914122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:13.914122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.914122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:13.914847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.914847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:13.914847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:13.914847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:13.914847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:13.914847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:13.914847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.914847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:13.914847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:13.914847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.914847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:13.914847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:13.914847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.914847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:13.914847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:13.914847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.915933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:27:13.915933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:27:13.915933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:27:13.915933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:13.915933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:13.915933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:13.915933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:13.915933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:13.915933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:13.915933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:13.915933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:13.915933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:13.915933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:13.915933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:13.915933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:13.915933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:13.915933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:13.915933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:13.916506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:13.916506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:13.916506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:13.916506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:13.916506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:13.916506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.916506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:13.924893 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:13.925256 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:13.925433 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:13.925631 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:13.926076 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:13.926421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.927195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:13.927195 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:13.927195 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:13.927195 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:13.927195 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:13.927195 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:13.927195 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.927195 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:13.928321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.928321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:27:13.928321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:13.928321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:13.928321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:13.928321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:13.928321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:13.928321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:13.928321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:13.928321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:13.928321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:13.928321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:27:13.928321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:13.928321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:27:13.928321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:27:13.928987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:13.928987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:27:13.928987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:13.928987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:27:13.928987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:27:13.928987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:13.928987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:27:13.928987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.928987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:27:13.929696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.929696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:13.929696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:13.929696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:13.929696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.929696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:27:13.929696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:27:13.929696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:13.929696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:13.929696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:13.929696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:13.930224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:13.930224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:13.930224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:13.930224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:13.930224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:13.930224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:13.930224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:13.930224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:13.930224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:13.930224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:13.930224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:13.930224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:13.930224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:13.930224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:13.930997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:13.930997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:13.930997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.930997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:13.931690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.931690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:13.931690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:13.931690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:13.931690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:13.931690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:13.931690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.931690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:13.931690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:13.931690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.931690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:13.931690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:13.931690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.931690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:13.931690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:13.931690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.931690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:27:13.932349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:27:13.932349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:13.932349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:13.932349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:13.932349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:13.932349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:13.932349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:13.932349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:13.932349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:13.932349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:13.932349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:13.932349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:13.932349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:13.932349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:13.932349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:13.932349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:13.932349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:13.932349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:13.932349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:13.932908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:13.932908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:13.932908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.932908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:13.933879 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:13.934172 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:13.934284 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:13.934427 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:13.934913 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:13.935297 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:13.935531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.936244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:13.936244 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:13.936244 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:13.936244 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:13.936244 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:13.936244 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:13.936244 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.936244 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:13.937028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.937028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:13.937028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:13.937028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:13.937028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.937028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:27:13.937028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:13.937028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:13.937028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:13.937028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:13.937028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:13.937028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:13.937396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:13.937396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:13.937396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:13.937396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:13.937396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:13.937396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:13.937396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:13.937396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:13.937396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:13.937396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:13.937396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:13.937396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:13.937396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:13.937784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.937784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:13.937976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.937976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:13.937976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:13.937976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:13.937976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:13.937976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:13.937976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.937976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:13.937976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:13.937976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.937976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:13.937976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:13.937976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.937976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:13.937976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:13.937976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:13.938444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:27:13.938444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:13.938444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:13.938444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:13.938444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:13.938444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:13.938444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:13.938444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:13.938444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:13.938444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:13.938444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:13.938444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:13.938444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:13.938444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:13.938444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:13.938444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:13.938444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:13.938444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:13.938444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:13.939034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:13.939034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:13.939034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:18.619308 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:18.620108 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:18.620108 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:18.620772 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:18.654949 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:18.654949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.655145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:18.655145 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:18.655145 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:18.655145 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:18.655145 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:18.655145 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:18.655145 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.655145 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:18.656090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.656090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:18.656090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:18.656090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:18.656090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.656090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:27:18.656090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:18.656090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:18.656090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:18.656090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:18.656090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:18.656090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:18.656711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:18.656711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:18.656711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:18.656711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:18.656711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:18.656711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:18.656711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:18.656711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:18.656711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:18.656711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:18.656711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:18.656711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:18.656711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:18.657303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.657303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:18.657587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.657587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:18.657587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:18.657587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:18.657587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:18.657587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:18.657587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.657587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:18.657587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:18.657587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.657587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:18.657587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:18.657587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.657587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:18.657587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:18.657587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.658154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:27:18.658154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:18.658154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:18.658154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:18.658154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:18.658154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:18.658154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:18.658154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:18.658154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:18.658154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:18.658154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:18.658154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:18.658154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:18.658154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:18.658154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:18.658154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:18.658154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:18.658154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:18.658154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:18.658826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:18.658826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.658826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:18.854828 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:18.855275 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:18.855389 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:18.856236 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:18.856546 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:18.856860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.857613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:18.857613 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:18.857613 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:18.857613 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:18.857613 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:18.857613 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:18.857613 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.857613 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:18.858544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.858544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:18.858544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:18.858544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:18.858544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.858544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:27:18.858544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:18.858544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:18.858544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:18.858544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:18.858544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:18.858544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:18.859182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:18.859182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:18.859182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:18.859182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:18.859182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:18.859182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:18.859182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:18.859182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:18.859182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:18.859182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:18.859182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:18.859182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:18.859182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:18.859798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.859798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:18.859798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.859798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:18.859798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:18.859798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:18.859798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:18.859798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:18.859798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.859798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:18.859798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:18.859798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.859798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:18.859798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:18.859798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.860480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:18.860480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:18.860480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.860480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:27:18.860480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:18.860480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:18.860480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:18.860480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:18.860480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:18.860480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:18.860480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:18.860480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:18.860480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:18.860480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:18.860480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:18.860480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:18.860480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:18.860480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:18.860480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:18.861411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:18.861411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:18.861411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:18.861411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:18.861411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.861411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:18.908881 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:18.909325 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:18.909499 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:18.909699 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:27:18.910508 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:18.910883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.911760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:18.911760 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:18.911760 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:18.911760 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:18.911760 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:18.911760 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:18.911760 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.911760 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:18.913041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.913041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:27:18.913041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:18.913041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:18.913041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:18.913041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:18.913041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:18.913041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:18.913041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:18.913041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:18.913041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:18.913041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:27:18.913041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:18.913041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:27:18.913041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:27:18.913775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:18.913775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:27:18.913775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:18.913775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:27:18.913775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:27:18.913775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:18.913775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:27:18.913775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.913775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:27:18.914224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.914224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:18.914224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:18.914224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:18.914224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.914224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:27:18.914224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:27:18.914224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:27:18.914224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:18.914224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:18.914224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:18.914224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:18.914836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:18.914836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:18.914836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:18.914836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:18.914836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:18.914836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:18.914836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:18.914836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:18.914836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:18.914836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:18.914836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:18.914836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:18.914836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:18.914836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:18.915502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:18.915502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:18.915502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.915502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:18.915502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.915502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:18.915502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:18.915502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:18.915502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:18.915502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:18.915502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.915502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:18.915502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:18.915502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.916160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:18.916160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:18.916160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.916160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:18.916160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:18.916160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.916160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:27:18.916160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:27:18.916160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:27:18.916160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:18.916160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:18.916160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:18.916160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:18.916160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:18.916160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:18.916160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:18.916160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:18.916160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:18.916160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:18.917079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:18.917079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:18.917079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:18.917079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:18.917079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:18.917079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:18.917079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:18.917079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:18.917079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:18.917079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:18.917079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.917079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:18.925326 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:18.925670 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:18.925872 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:18.926060 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:18.926496 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:18.926883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.927561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:18.927561 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:18.927561 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:18.927561 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:18.927561 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:18.927561 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:18.927561 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.927561 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:18.928797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.928797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:27:18.928797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:18.928797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:18.928797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:18.928797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:18.928797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:18.928797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:18.928797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:18.928797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:18.928797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:18.928797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:27:18.928797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:18.928797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:27:18.928797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:27:18.929482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:18.929482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:27:18.929482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:18.929482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:27:18.929482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:27:18.929482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:18.929482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:27:18.929482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.929482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:27:18.930134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.930134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:18.930134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:18.930134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:18.930134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.930134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:27:18.930134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:27:18.930134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:18.930134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:18.930134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:18.930134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:18.930623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:18.930623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:18.930623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:18.930623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:18.930623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:18.930623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:18.930623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:18.930623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:18.930623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:18.930623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:18.930623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:18.930623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:18.930623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:18.930623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:18.931228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:18.931228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:18.931228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.931228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:18.931774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.931774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:18.931774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:18.931774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:18.931774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:18.931774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:18.931774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.931774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:18.931774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:18.931774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.931774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:18.931774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:18.931774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.931774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:18.931774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:18.931774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.931774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:27:18.932531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:27:18.932531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:18.932531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:18.932531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:18.932531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:18.932531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:18.932531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:18.932531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:18.932531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:18.932531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:18.932531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:18.932531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:18.932531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:18.932531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:18.932531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:18.932531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:18.932531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:18.932531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:18.932531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:18.933328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:18.933328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:18.933328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.933328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:18.934368 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:18.934642 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:18.934785 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:18.934963 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:18.935374 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:18.935745 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:18.935968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.936627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:18.936627 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:18.936627 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:18.936627 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:18.936627 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:18.936627 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:18.936627 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.936627 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:18.937421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.937421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:18.937421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:18.937421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:18.937421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.937421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:27:18.937421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:18.937421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:18.937421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:18.937421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:18.937421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:18.937421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:18.937757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:18.937757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:18.937757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:18.937757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:18.937757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:18.937757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:18.937757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:18.937757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:18.937757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:18.937757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:18.937757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:18.937757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:18.937757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:18.938094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.938094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:18.938351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.938351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:18.938351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:18.938351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:18.938351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:18.938351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:18.938351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.938351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:18.938351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:18.938351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.938351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:18.938351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:18.938351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.938351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:18.938351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:18.938351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:18.938793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:27:18.938793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:18.938793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:18.938793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:18.938793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:18.938793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:18.938793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:18.938793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:18.938793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:18.938793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:18.938793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:18.938793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:18.938793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:18.938793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:18.938793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:18.938793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:18.938793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:18.938793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:18.938793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:18.939261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:18.939261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:18.939261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:23.620092 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:23.621240 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:23.621240 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:23.622778 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:23.661926 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:23.662266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.663051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:23.663051 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:23.663051 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:23.663051 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:23.663051 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:23.663051 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:23.663051 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.663051 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:23.664000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.664000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:23.664000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:23.664000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:23.664000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.664000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:27:23.664000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:23.664000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:23.664000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:23.664000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:23.664000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:23.664000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:23.665009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:23.665009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:23.665009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:23.665009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:23.665009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:23.665009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:23.665009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:23.665009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:23.665009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:23.665009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:23.665009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:23.665009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:23.665009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:23.665565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.665565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:23.665565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.665565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:23.665565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:23.665565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:23.665565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:23.665565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:23.665565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.665565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:23.665565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:23.665565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.665565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:23.665565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:23.665565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.666337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:23.666337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:23.666337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.666337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:27:23.666337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:23.666337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:23.666337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:23.666337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:23.666337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:23.666337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:23.666337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:23.666337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:23.666337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:23.666337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:23.666337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:23.666337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:23.666337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:23.666337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:23.666337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:23.668044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:23.668044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:23.668044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:23.668044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:23.668044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.668044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:23.855309 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:23.855969 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:23.856169 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:23.857297 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:23.857713 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:23.857974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.858976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:23.858976 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:23.858976 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:23.858976 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:23.858976 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:23.858976 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:23.858976 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.858976 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:23.860844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.860844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:23.860844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:23.860844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:23.860844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.860844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:27:23.860844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:23.860844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:23.860844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:23.860844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:23.860844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:23.860844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:23.861842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:23.861842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:23.861842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:23.861842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:23.861842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:23.861842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:23.861842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:23.861842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:23.861842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:23.861842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:23.861842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:23.861842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:23.861842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:23.862527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.862527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:23.862716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.862716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:23.862716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:23.862716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:23.862716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:23.862716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:23.862716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.862716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:23.862716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:23.862716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.862716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:23.862716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:23.862716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.862716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:23.862716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:23.862716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.863569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:27:23.863569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:23.863569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:23.863569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:23.863569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:23.863569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:23.863569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:23.863569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:23.863569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:23.863569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:23.863569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:23.863569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:23.863569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:23.863569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:23.863569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:23.863569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:23.863569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:23.863569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:23.863569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:23.865537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:23.865537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.865537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:23.909436 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:23.910128 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:23.910337 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:23.910544 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:27:23.911349 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:23.911668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.912588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:23.912588 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:23.912588 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:23.912588 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:23.912588 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:23.912588 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:23.912588 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.912588 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:23.913451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.913451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:27:23.913451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:23.913451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:23.913451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:23.913451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:23.913451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:23.913451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:23.913451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:23.913451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:23.913451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:23.913451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:27:23.913451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:23.913451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:27:23.913451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:27:23.914476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:23.914476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:27:23.914476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:23.914476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:27:23.914476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:27:23.914476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:23.914476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:27:23.914476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.914476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:27:23.914476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.914476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:23.914476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:23.914476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:23.914476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.915152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:27:23.915152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:27:23.915152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:27:23.915152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:23.915152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:23.915152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:23.915152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:23.915152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:23.915152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:23.915152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:23.915152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:23.915152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:23.915152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:23.915763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:23.915763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:23.915763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:23.915763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:23.915763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:23.915763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:23.915763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:23.915763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:23.915763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:23.915763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:23.915763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.915763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:23.916266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.916266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:23.916266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:23.916266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:23.916266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:23.916266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:23.916266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.916266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:23.916266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:23.916266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.916266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:23.916266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:23.916266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.916266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:23.916266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:23.916266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.918402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:27:23.918402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:27:23.918402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:27:23.918402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:23.918402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:23.918402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:23.918402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:23.918402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:23.918402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:23.918402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:23.918402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:23.918402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:23.918402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:23.918402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:23.918402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:23.918402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:23.918402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:23.918402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:23.919437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:23.919437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:23.919437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:23.919437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:23.919437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:23.919437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.919437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:23.925731 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:23.926000 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:23.926156 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:23.926268 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:23.926827 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:23.927236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.927975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:23.927975 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:23.927975 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:23.927975 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:23.927975 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:23.927975 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:23.927975 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.927975 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:23.928938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.928938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:27:23.928938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:23.928938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:23.928938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:23.928938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:23.928938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:23.928938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:23.928938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:23.928938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:23.928938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:23.928938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:27:23.928938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:23.928938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:27:23.928938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:27:23.930444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:23.930444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:27:23.930444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:23.930444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:27:23.930444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:27:23.930444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:23.930444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:27:23.930444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.930444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:27:23.930875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.930875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:23.930875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:23.930875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:23.930875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.930875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:27:23.930875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:27:23.930875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:23.930875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:23.930875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:23.930875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:23.931394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:23.931394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:23.931394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:23.931394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:23.931394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:23.931394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:23.931394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:23.931394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:23.931394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:23.931394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:23.931394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:23.931394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:23.931394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:23.931394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:23.931983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:23.931983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:23.931983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.931983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:23.932761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.932761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:23.932761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:23.932761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:23.932761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:23.932761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:23.932761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.932761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:23.932761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:23.932761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.932761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:23.932761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:23.932761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.932761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:23.932761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:23.932761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.932761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:27:23.934540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:27:23.934540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:23.934540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:23.934540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:23.934540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:23.934540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:23.934540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:23.934540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:23.934540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:23.934540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:23.934540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:23.934540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:23.934540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:23.934540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:23.934540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:23.934540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:23.934540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:23.934540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:23.934540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:23.935998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:23.935998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:23.935998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.935998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:23.935998 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:23.935998 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:23.935998 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:23.935998 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:23.936936 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:23.936936 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:23.936936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.937344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:23.937344 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:23.937344 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:23.937344 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:23.937344 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:23.937344 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:23.937344 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.937344 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:23.938042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.938042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:23.938042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:23.938042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:23.938042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.938042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:27:23.938042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:23.938042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:23.938042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:23.938042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:23.938042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:23.938042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:23.938507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:23.938507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:23.938507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:23.938507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:23.938507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:23.938507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:23.938507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:23.938507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:23.938507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:23.938507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:23.938507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:23.938507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:23.938507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:23.938742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.938742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:23.939074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.939074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:23.939074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:23.939074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:23.939074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:23.939074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:23.939074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.939074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:23.939074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:23.939074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.939074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:23.939074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:23.939074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.939074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:23.939074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:23.939074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:23.939717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:27:23.939717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:23.939717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:23.939717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:23.939717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:23.939717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:23.939717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:23.939717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:23.939717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:23.939717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:23.939717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:23.939717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:23.939717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:23.939717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:23.939717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:23.939717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:23.939717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:23.939717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:23.939717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:23.940552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:23.940552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:23.940552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:28.621027 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:28.621697 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:28.621697 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:28.622733 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:28.656219 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:28.656385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.656857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:28.656857 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:28.656857 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:28.656857 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:28.656857 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:28.656857 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:28.656857 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.656857 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:28.657597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.657597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:28.657597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:28.657597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:28.657597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.657597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:27:28.657597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:28.657597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:28.657597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:28.657597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:28.657597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:28.657597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:28.658592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:28.658592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:28.658592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:28.658592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:28.658592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:28.658592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:28.658592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:28.658592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:28.658592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:28.658592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:28.658592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:28.658592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:28.658592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:28.659021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.659021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:28.659021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.659021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:28.659021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:28.659021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:28.659021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:28.659021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:28.659021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.659021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:28.659021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:28.659021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.659021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:28.659021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:28.659021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.659398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:28.659398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:28.659398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.659398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:27:28.659398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:28.659398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:28.659398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:28.659398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:28.659398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:28.659398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:28.659398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:28.659398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:28.659398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:28.659398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:28.659398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:28.659398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:28.659398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:28.659398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:28.659398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:28.659904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:28.659904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:28.659904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:28.659904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:28.659904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.659904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:28.856112 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:28.856819 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:28.857109 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:28.858114 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:28.858490 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:28.858872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.860039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:28.860039 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:28.860039 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:28.860039 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:28.860039 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:28.860039 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:28.860039 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.860039 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:28.861432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.861432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:28.861432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:28.861432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:28.861432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.861432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:27:28.861432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:28.861432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:28.861432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:28.861432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:28.861432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:28.861432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:28.861971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:28.861971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:28.861971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:28.861971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:28.861971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:28.861971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:28.861971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:28.861971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:28.861971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:28.861971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:28.861971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:28.861971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:28.861971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:28.862398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.862398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:28.863060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.863060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:28.863060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:28.863060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:28.863060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:28.863060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:28.863060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.863060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:28.863060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:28.863060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.863060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:28.863060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:28.863060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.863060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:28.863060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:28.863060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.863514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:27:28.863514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:28.863514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:28.863514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:28.863514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:28.863514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:28.863514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:28.863514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:28.863514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:28.863514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:28.863514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:28.863514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:28.863514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:28.863514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:28.863514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:28.863514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:28.863514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:28.863514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:28.863514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:28.864016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:28.864016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.864016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:28.910095 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:28.910512 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:28.910834 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:28.911151 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:27:28.911662 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:28.912104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.913306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:28.913306 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:28.913306 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:28.913306 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:28.913306 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:28.913306 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:28.913306 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.913306 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:28.915308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.915308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:27:28.915308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:28.915308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:28.915308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:28.915308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:28.915308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:28.915308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:28.915308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:28.915308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:28.915308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:28.915308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:27:28.915308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:28.915308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:27:28.915308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:27:28.916083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:28.916083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:27:28.916083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:28.916083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:27:28.916083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:27:28.916083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:28.916083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:27:28.916083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.916083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:27:28.917074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.917074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:28.917074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:28.917074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:28.917074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.917074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:27:28.917074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:27:28.917074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:27:28.917074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:28.917074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:28.917074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:28.917074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:28.917410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:28.917410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:28.917410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:28.917410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:28.917410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:28.917410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:28.917410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:28.917410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:28.917410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:28.917410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:28.917410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:28.917410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:28.917410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:28.917410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:28.917798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:28.917798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:28.917798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.917798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:28.918277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.918277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:28.918277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:28.918277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:28.918277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:28.918277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:28.918277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.918277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:28.918277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:28.918277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.918277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:28.918277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:28.918277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.918277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:28.918277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:28.918277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.918725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:27:28.918725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:27:28.918725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:27:28.918725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:28.918725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:28.918725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:28.918725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:28.918725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:28.918725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:28.918725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:28.918725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:28.918725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:28.918725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:28.918725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:28.918725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:28.918725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:28.918725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:28.918725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:28.919164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:28.919164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:28.919164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:28.919164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:28.919164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:28.919164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.919164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:28.926132 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:28.926443 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:28.926561 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:28.926761 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:28.927312 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:28.927799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.928591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:28.928591 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:28.928591 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:28.928591 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:28.928591 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:28.928591 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:28.928591 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.928591 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:28.929379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.929379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:27:28.929379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:28.929379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:28.929379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:28.929379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:28.929379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:28.929379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:28.929379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:28.929379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:28.929379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:28.929379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:27:28.929379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:28.929379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:27:28.929379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:27:28.930226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:28.930226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:27:28.930226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:28.930226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:27:28.930226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:27:28.930226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:28.930226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:27:28.930226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.930226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:27:28.930843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.930843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:28.930843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:28.930843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:28.930843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.930843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:27:28.930843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:27:28.930843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:28.930843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:28.930843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:28.930843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:28.931317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:28.931317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:28.931317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:28.931317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:28.931317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:28.931317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:28.931317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:28.931317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:28.931317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:28.931317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:28.931317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:28.931317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:28.931317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:28.931317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:28.931928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:28.931928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:28.931928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.931928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:28.932095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.932095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:28.932095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:28.932095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:28.932095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:28.932095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:28.932095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.932095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:28.932095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:28.932095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.932095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:28.932095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:28.932095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.932095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:28.932095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:28.932095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.932095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:27:28.932797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:27:28.932797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:28.932797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:28.932797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:28.932797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:28.932797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:28.932797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:28.932797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:28.932797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:28.932797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:28.932797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:28.932797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:28.932797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:28.932797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:28.932797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:28.932797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:28.932797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:28.932797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:28.932797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:28.934259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:28.934259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:28.934259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.934259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:28.935442 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:28.935798 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:28.935913 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:28.936070 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:28.936409 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:28.936692 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:28.936899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.937434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:28.937434 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:28.937434 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:28.937434 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:28.937434 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:28.937434 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:28.937434 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.937434 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:28.938388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.938388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:28.938388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:28.938388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:28.938388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.938388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:27:28.938388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:28.938388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:28.938388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:28.938388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:28.938388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:28.938388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:28.939079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:28.939079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:28.939079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:28.939079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:28.939079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:28.939079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:28.939079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:28.939079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:28.939079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:28.939079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:28.939079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:28.939079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:28.939079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:28.939493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.939493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:28.939493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.939493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:28.939493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:28.939493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:28.939493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:28.939493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:28.939493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.939493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:28.939493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:28.939493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.939493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:28.939493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:28.939493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:27:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:28.940580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:28.940580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:28.940580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:28.940580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:28.940580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:28.940580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:33.621838 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:33.622727 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:33.622727 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:33.623903 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:33.655654 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:33.656124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.657103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:33.657103 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:33.657103 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:33.657103 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:33.657103 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:33.657103 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:33.657103 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.657103 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:33.658166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.658166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:33.658166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:33.658166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:33.658166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.658166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:27:33.658166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:33.658166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:33.658166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:33.658166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:33.658166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:33.658166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:33.658977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:33.658977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:33.658977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:33.658977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:33.658977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:33.658977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:33.658977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:33.658977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:33.658977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:33.658977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:33.658977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:33.658977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:33.658977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:33.659596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.659596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:33.659596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.659596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:33.659596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:33.659596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:33.659596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:33.659596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:33.659596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.659596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:33.659596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:33.659596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.659596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:33.659596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:33.659596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.660352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:33.660352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:33.660352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.660352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:27:33.660352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:33.660352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:33.660352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:33.660352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:33.660352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:33.660352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:33.660352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:33.660352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:33.660352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:33.660352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:33.660352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:33.660352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:33.660352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:33.660352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:33.660352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:33.661486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:33.661486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:33.661486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:33.661486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:33.661486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.661486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:33.856777 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:33.857196 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:33.857396 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:33.858442 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:33.858886 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:33.859188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.860177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:33.860177 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:33.860177 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:33.860177 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:33.860177 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:33.860177 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:33.860177 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.860177 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:33.861645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.861645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:33.861645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:33.861645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:33.861645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.861645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:27:33.861645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:33.861645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:33.861645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:33.861645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:33.861645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:33.861645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:33.862214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:33.862214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:33.862214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:33.862214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:33.862214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:33.862214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:33.862214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:33.862214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:33.862214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:33.862214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:33.862214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:33.862214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:33.862214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:33.862780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.862780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:33.863076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.863076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:33.863076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:33.863076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:33.863076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:33.863076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:33.863076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.863076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:33.863076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:33.863076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.863076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:33.863076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:33.863076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.863076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:33.863076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:33.863076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.863798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:27:33.863798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:33.863798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:33.863798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:33.863798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:33.863798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:33.863798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:33.863798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:33.863798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:33.863798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:33.863798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:33.863798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:33.863798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:33.863798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:33.863798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:33.863798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:33.863798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:33.863798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:33.863798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:33.864578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:33.864578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.864578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:33.910495 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:33.911029 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:33.911231 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:33.911421 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:27:33.912271 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:33.912630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.913600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:33.913600 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:33.913600 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:33.913600 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:33.913600 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:33.913600 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:33.913600 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.913600 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:33.914979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.914979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:27:33.914979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:33.914979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:33.914979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:33.914979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:33.914979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:33.914979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:33.914979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:33.914979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:33.914979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:33.914979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:27:33.914979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:33.914979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:27:33.914979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:27:33.915721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:33.915721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:27:33.915721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:33.915721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:27:33.915721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:27:33.915721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:33.915721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:27:33.915721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.915721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:27:33.916391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.916391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:33.916391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:33.916391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:33.916391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.916391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:27:33.916391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:27:33.916391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:27:33.916391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:33.916391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:33.916391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:33.916391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:33.916994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:33.916994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:33.916994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:33.916994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:33.916994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:33.916994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:33.916994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:33.916994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:33.916994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:33.916994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:33.916994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:33.916994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:33.916994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:33.916994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:33.917595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:33.917595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:33.917595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.917595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:33.918520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.918520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:33.918520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:33.918520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:33.918520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:33.918520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:33.918520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.918520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:33.918520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:33.918520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.918520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:33.918520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:33.918520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.918520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:33.918520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:33.918520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.919255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:27:33.919255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:27:33.919255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:27:33.919255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:33.919255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:33.919255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:33.919255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:33.919255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:33.919255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:33.919255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:33.919255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:33.919255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:33.919255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:33.919255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:33.919255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:33.919255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:33.919255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:33.919255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:33.920050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:33.920050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:33.920050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:33.920050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:33.920050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:33.920050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.920050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:33.926481 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:33.926758 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:33.926873 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:33.926995 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:33.927354 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:33.927586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.928206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:33.928206 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:33.928206 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:33.928206 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:33.928206 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:33.928206 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:33.928206 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.928206 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:33.929553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.929553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:27:33.929553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:33.929553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:33.929553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:33.929553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:33.929553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:33.929553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:33.929553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:33.929553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:33.929553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:33.929553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:27:33.929553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:33.929553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:27:33.929553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:27:33.929972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:33.929972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:27:33.929972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:33.929972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:27:33.929972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:27:33.929972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:33.929972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:27:33.929972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.929972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:27:33.931220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.931220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:33.931220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:33.931220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:33.931220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.931220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:27:33.931220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:27:33.931220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:33.931220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:33.931220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:33.931220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:33.931533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:33.931533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:33.931533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:33.931533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:33.931533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:33.931533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:33.931533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:33.931533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:33.931533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:33.931533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:33.931533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:33.931533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:33.931533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:33.931533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:33.932067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:33.932067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:33.932067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.932067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:33.933216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.933216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:33.933216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:33.933216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:33.933216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:33.933216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:33.933216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.933216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:33.933216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:33.933216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.933216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:33.933216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:33.933216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.933216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:33.933216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:33.933216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.933216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:27:33.933999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:27:33.933999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:33.933999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:33.933999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:33.933999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:33.933999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:33.933999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:33.933999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:33.933999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:33.933999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:33.933999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:33.933999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:33.933999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:33.933999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:33.933999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:33.933999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:33.933999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:33.933999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:33.933999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:33.934804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:33.934804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:33.934804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.934804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:33.935836 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:33.936176 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:33.936349 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:33.936588 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:33.937087 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:33.937631 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:33.937992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.938739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:33.938739 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:33.938739 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:33.938739 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:33.938739 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:33.938739 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:33.938739 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.938739 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:33.939581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.939581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:33.939581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:33.939581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:33.939581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.939581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:27:33.939581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:33.939581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:33.939581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:33.939581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:33.939581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:33.939581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:33.940255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:33.940255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:33.940255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:33.940255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:33.940255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:33.940255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:33.940255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:33.940255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:33.940255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:33.940255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:33.940255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:33.940255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:33.940255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:33.940893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.940893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:33.940893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.940893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:33.940893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:33.940893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:33.940893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:33.940893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:33.940893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.940893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:33.940893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:33.940893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.940893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:33.940893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:33.940893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.941400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:33.941400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:33.941400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:33.941400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:27:33.941400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:33.941400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:33.941400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:33.941400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:33.941400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:33.941400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:33.941400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:33.941400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:33.941400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:33.941400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:33.941400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:33.941400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:33.941400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:33.941400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:33.941400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:33.942174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:33.942174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:33.942174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:33.942174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:33.942174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:33.942174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:38.622475 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:38.623577 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:38.623577 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:38.624796 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:38.661416 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:38.661941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.662702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:38.662702 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:38.662702 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:38.662702 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:38.662702 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:38.662702 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:38.662702 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.662702 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:38.663759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.663759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:38.663759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:38.663759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:38.663759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.663759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:27:38.663759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:38.663759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:38.663759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:38.663759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:38.663759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:38.663759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:38.664464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:38.664464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:38.664464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:38.664464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:38.664464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:38.664464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:38.664464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:38.664464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:38.664464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:38.664464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:38.664464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:38.664464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:38.664464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:38.665173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.665173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:38.665173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.665173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:38.665173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:38.665173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:38.665173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:38.665173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:38.665173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.665173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:38.665173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:38.665173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.665173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:38.665173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:38.665173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.666021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:38.666021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:38.666021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.666021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:27:38.666021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:38.666021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:38.666021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:38.666021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:38.666021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:38.666021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:38.666021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:38.666021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:38.666021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:38.666021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:38.666021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:38.666021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:38.666021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:38.666021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:38.666021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:38.667569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:38.667569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:38.667569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:38.667569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:38.667569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.667569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:38.857228 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:38.858035 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:38.858222 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:38.859559 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:38.859881 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:38.860080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.861743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:38.861743 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:38.861743 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:38.861743 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:38.861743 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:38.861743 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:38.861743 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.861743 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:38.862774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.862774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:38.862774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:38.862774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:38.862774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.862774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:27:38.862774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:38.862774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:38.862774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:38.862774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:38.862774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:38.862774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:38.863410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:38.863410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:38.863410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:38.863410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:38.863410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:38.863410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:38.863410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:38.863410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:38.863410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:38.863410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:38.863410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:38.863410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:38.863410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:38.864116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.864116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:38.864723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.864723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:38.864723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:38.864723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:38.864723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:38.864723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:38.864723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.864723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:38.864723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:38.864723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.864723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:38.864723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:38.864723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.864723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:38.864723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:38.864723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.866305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:27:38.866305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:38.866305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:38.866305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:38.866305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:38.866305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:38.866305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:38.866305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:38.866305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:38.866305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:38.866305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:38.866305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:38.866305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:38.866305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:38.866305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:38.866305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:38.866305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:38.866305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:38.866305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:38.868104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:38.868104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.868104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:38.911158 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:38.911743 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:38.911893 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:38.912065 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:27:38.912992 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:38.913358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.914346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:38.914346 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:38.914346 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:38.914346 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:38.914346 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:38.914346 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:38.914346 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.914346 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:38.915620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.915620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:27:38.915620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:38.915620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:38.915620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:38.915620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:38.915620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:38.915620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:38.915620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:38.915620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:38.915620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:38.915620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:27:38.915620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:38.915620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:27:38.915620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:27:38.916398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:38.916398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:27:38.916398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:38.916398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:27:38.916398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:27:38.916398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:38.916398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:27:38.916398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.916398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:27:38.917062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.917062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:38.917062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:38.917062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:38.917062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.917062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:27:38.917062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:27:38.917062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:27:38.917062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:38.917062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:38.917062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:38.917062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:38.918581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:38.918581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:38.918581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:38.918581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:38.918581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:38.918581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:38.918581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:38.918581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:38.918581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:38.918581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:38.918581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:38.918581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:38.918581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:38.918581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:38.919889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:38.919889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:38.919889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.919889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:38.919889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.919889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:38.919889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:38.919889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:38.919889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:38.919889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:38.919889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.919889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:38.919889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:38.919889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.920524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:38.920524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:38.920524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.920524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:38.920524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:38.920524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.920524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:27:38.920524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:27:38.920524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:27:38.920524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:38.920524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:38.920524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:38.920524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:38.920524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:38.920524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:38.920524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:38.920524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:38.920524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:38.920524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:38.921382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:38.921382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:38.921382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:38.921382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:38.921382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:38.921382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:38.921382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:38.921382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:38.921382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:38.921382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:38.921382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.921382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:38.926965 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:38.927451 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:38.927656 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:38.927910 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:38.928454 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:38.928851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.929882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:38.929882 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:38.929882 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:38.929882 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:38.929882 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:38.929882 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:38.929882 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.929882 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:38.931316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.931316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:27:38.931316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:38.931316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:38.931316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:38.931316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:38.931316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:38.931316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:38.931316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:38.931316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:38.931316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:38.931316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:27:38.931316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:38.931316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:27:38.931316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:27:38.932386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:38.932386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:27:38.932386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:38.932386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:27:38.932386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:27:38.932386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:38.932386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:27:38.932386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.932386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:27:38.933107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.933107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:38.933107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:38.933107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:38.933107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.933107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:27:38.933107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:27:38.933107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:38.933107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:38.933107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:38.933107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:38.933569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:38.933569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:38.933569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:38.933569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:38.933569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:38.933569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:38.933569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:38.933569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:38.933569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:38.933569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:38.933569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:38.933569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:38.933569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:38.933569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:38.934164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:38.934164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:38.934164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.934164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:38.934837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.934837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:38.934837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:38.934837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:38.934837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:38.934837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:38.934837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.934837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:38.934837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:38.934837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.934837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:38.934837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:38.934837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.934837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:38.934837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:38.934837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.934837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:27:38.935545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:27:38.935545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:38.935545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:38.935545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:38.935545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:38.935545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:38.935545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:38.935545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:38.935545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:38.935545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:38.935545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:38.935545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:38.935545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:38.935545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:38.935545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:38.935545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:38.935545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:38.935545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:38.935545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:38.936312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:38.936312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:38.936312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.936312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:38.936312 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:38.936630 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:38.936630 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:38.936774 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:38.937290 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:38.937632 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:38.937987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.938732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:38.938732 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:38.938732 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:38.938732 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:38.938732 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:38.938732 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:38.938732 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.938732 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:38.939427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.939427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:38.939427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:38.939427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:38.939427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.939427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:27:38.939427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:38.939427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:38.939427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:38.939427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:38.939427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:38.939427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:38.940010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:38.940010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:38.940010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:38.940010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:38.940010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:38.940010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:38.940010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:38.940010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:38.940010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:38.940010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:38.940010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:38.940010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:38.940010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:38.940360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.940360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:38.940360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.940360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:38.940360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:38.940360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:38.940360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:38.940360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:38.940360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.940360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:38.940360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:38.940360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.940360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:38.940360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:38.940360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.940829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:38.940829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:38.940829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:38.940829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:27:38.940829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:38.940829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:38.940829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:38.940829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:38.940829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:38.940829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:38.940829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:38.940829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:38.940829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:38.940829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:38.940829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:38.940829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:38.940829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:38.940829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:38.940829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:38.941332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:38.941332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:38.941332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:38.941332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:38.941332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:38.941332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:43.623078 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:43.623921 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:43.623921 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:43.625150 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:43.657257 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:43.657686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.658524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:43.658524 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:43.658524 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:43.658524 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:43.658524 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:43.658524 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:43.658524 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.658524 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:43.659521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.659521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:43.659521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:43.659521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:43.659521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.659521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:27:43.659521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:43.659521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:43.659521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:43.659521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:43.659521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:43.659521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:43.659884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:43.659884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:43.659884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:43.659884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:43.659884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:43.659884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:43.659884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:43.659884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:43.659884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:43.659884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:43.659884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:43.659884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:43.659884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:43.660438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.660438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:43.660840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.660840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:43.660840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:43.660840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:43.660840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:43.660840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:43.660840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.660840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:43.660840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:43.660840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.660840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:43.660840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:43.660840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.660840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:43.660840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:43.660840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.661573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:27:43.661573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:43.661573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:43.661573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:43.661573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:43.661573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:43.661573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:43.661573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:43.661573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:43.661573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:43.661573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:43.661573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:43.661573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:43.661573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:43.661573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:43.661573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:43.661573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:43.661573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:43.661573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:43.662397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:43.662397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.662397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:43.860940 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:43.860940 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:43.860940 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:43.863732 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:43.863732 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:43.863732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.863732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:43.863732 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:43.863732 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:43.863732 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:43.863732 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:43.863732 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:43.863732 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.863732 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:43.866490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.866490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:43.866490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:43.866490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:43.866490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.866490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:27:43.866490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:43.866490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:43.866490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:43.866490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:43.866490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:43.866490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:43.867953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:43.867953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:43.867953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:43.867953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:43.867953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:43.867953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:43.867953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:43.867953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:43.867953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:43.867953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:43.867953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:43.867953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:43.867953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:43.868593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.868593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:43.868593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.868593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:43.868593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:43.868593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:43.868593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:43.868593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:43.868593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.868593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:43.868593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:43.868593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.868593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:43.868593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:43.868593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.869320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:43.869320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:43.869320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.869320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:27:43.869320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:43.869320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:43.869320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:43.869320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:43.869320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:43.869320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:43.869320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:43.869320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:43.869320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:43.869320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:43.869320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:43.869320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:43.869320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:43.869320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:43.869320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:43.872152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:43.872152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:43.872152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:43.872152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:43.872152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.872152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:43.911787 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:43.912328 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:43.912583 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:43.912845 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:27:43.913521 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:43.913793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.914510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:43.914510 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:43.914510 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:43.914510 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:43.914510 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:43.914510 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:43.914510 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.914510 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:43.916073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.916073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:27:43.916073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:43.916073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:43.916073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:43.916073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:43.916073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:43.916073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:43.916073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:43.916073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:43.916073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:43.916073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:27:43.916073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:43.916073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:27:43.916073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:27:43.916370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:43.916370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:27:43.916370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:43.916370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:27:43.916370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:27:43.916370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:43.916370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:27:43.916370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.916370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:27:43.917446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.917446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:43.917446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:43.917446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:43.917446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.917446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:27:43.917446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:27:43.917446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:27:43.917446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:43.917446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:43.917446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:43.917446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:43.917700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:43.917700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:43.917700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:43.917700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:43.917700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:43.917700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:43.917700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:43.917700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:43.917700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:43.917700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:43.917700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:43.917700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:43.917700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:43.917700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:43.917944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:43.917944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:43.917944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.917944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:43.919414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.919414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:43.919414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:43.919414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:43.919414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:43.919414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:43.919414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.919414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:43.919414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:43.919414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.919414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:43.919414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:43.919414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.919414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:43.919414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:43.919414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.919720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:27:43.919720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:27:43.919720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:27:43.919720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:43.919720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:43.919720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:43.919720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:43.919720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:43.919720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:43.919720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:43.919720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:43.919720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:43.919720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:43.919720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:43.919720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:43.919720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:43.919720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:43.919720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:43.920046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:43.920046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:43.920046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:43.920046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:43.920046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:43.920046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.920046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:43.927457 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:43.927953 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:43.928173 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:43.928421 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:43.929143 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:43.929618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.930593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:43.930593 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:43.930593 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:43.930593 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:43.930593 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:43.930593 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:43.930593 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.930593 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:43.931963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.931963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:27:43.931963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:43.931963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:43.931963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:43.931963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:43.931963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:43.931963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:43.931963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:43.931963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:43.931963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:43.931963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:27:43.931963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:43.931963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:27:43.931963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:27:43.932227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:43.932227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:27:43.932227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:43.932227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:27:43.932227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:27:43.932227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:43.932227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:27:43.932227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.932227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:27:43.933556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.933556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:43.933556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:43.933556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:43.933556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.933556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:27:43.933556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:27:43.933556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:43.933556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:43.933556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:43.933556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:43.933767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:43.933767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:43.933767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:43.933767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:43.933767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:43.933767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:43.933767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:43.933767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:43.933767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:43.933767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:43.933767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:43.933767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:43.933767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:43.933767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:43.934009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:43.934009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:43.934009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.934009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:43.935665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.935665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:43.935665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:43.935665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:43.935665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:43.935665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:43.935665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.935665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:43.935665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:43.935665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.935665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:43.935665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:43.935665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.935665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:43.935665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:43.935665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.935665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:27:43.935993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:27:43.935993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:43.935993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:43.935993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:43.935993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:43.935993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:43.935993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:43.935993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:43.935993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:43.935993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:43.935993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:43.935993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:43.935993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:43.935993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:43.935993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:43.935993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:43.935993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:43.935993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:43.935993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:43.936305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:43.936305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:43.936305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.936305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:43.936769 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:43.937107 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:43.937223 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:43.937379 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:43.937838 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:43.938223 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:43.938454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.939185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:43.939185 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:43.939185 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:43.939185 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:43.939185 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:43.939185 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:43.939185 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.939185 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:43.940314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.940314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:43.940314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:43.940314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:43.940314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.940314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:27:43.940314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:43.940314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:43.940314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:43.940314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:43.940314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:43.940314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:43.940555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:43.940555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:43.940555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:43.940555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:43.940555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:43.940555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:43.940555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:43.940555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:43.940555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:43.940555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:43.940555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:43.940555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:43.940555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:43.940801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.940801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:43.941833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.941833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:43.941833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:43.941833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:43.941833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:43.941833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:43.941833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.941833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:43.941833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:43.941833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.941833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:43.941833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:43.941833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.941833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:43.941833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:43.941833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:43.942121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:27:43.942121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:43.942121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:43.942121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:43.942121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:43.942121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:43.942121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:43.942121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:43.942121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:43.942121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:43.942121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:43.942121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:43.942121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:43.942121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:43.942121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:43.942121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:43.942121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:43.942121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:43.942121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:43.942431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:43.942431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:43.942431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:48.623740 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:48.624671 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:48.624911 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:48.626580 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:48.661722 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:48.662150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.663342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:48.663342 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:48.663342 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:48.663342 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:48.663342 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:48.663342 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:48.663342 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.663342 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:48.664531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.664531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:48.664531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:48.664531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:48.664531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.664531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:27:48.664531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:48.664531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:48.664531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:48.664531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:48.664531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:48.664531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:48.664984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:48.664984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:48.664984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:48.664984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:48.664984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:48.664984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:48.664984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:48.664984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:48.664984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:48.664984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:48.664984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:48.664984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:48.664984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:48.665372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.665372 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:48.665723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.665723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:48.665723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:48.665723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:48.665723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:48.665723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:48.665723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.665723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:48.665723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:48.665723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.665723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:48.665723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:48.665723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.665723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:48.665723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:48.665723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.666246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:27:48.666246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:48.666246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:48.666246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:48.666246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:48.666246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:48.666246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:48.666246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:48.666246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:48.666246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:48.666246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:48.666246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:48.666246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:48.666246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:48.666246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:48.666246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:48.666246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:48.666246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:48.666246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:48.666847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:48.666847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.666847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:48.858562 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:48.859441 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:48.859748 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:48.861308 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:48.861651 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:48.861891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.862923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:48.862923 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:48.862923 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:48.862923 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:48.862923 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:48.862923 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:48.862923 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.862923 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:48.864554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.864554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:48.864554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:48.864554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:48.864554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.864554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:27:48.864554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:48.864554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:48.864554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:48.864554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:48.864554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:48.864554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:48.865158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:48.865158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:48.865158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:48.865158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:48.865158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:48.865158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:48.865158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:48.865158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:48.865158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:48.865158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:48.865158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:48.865158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:48.865158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:48.865732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.865732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:48.867103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.867103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:48.867103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:48.867103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:48.867103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:48.867103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:48.867103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.867103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:48.867103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:48.867103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.867103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:48.867103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:48.867103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.867103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:48.867103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:48.867103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.867850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:27:48.867850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:48.867850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:48.867850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:48.867850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:48.867850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:48.867850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:48.867850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:48.867850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:48.867850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:48.867850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:48.867850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:48.867850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:48.867850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:48.867850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:48.867850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:48.867850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:48.867850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:48.867850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:48.868666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:48.868666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.868666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:48.912514 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:48.913632 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:48.913876 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:48.914135 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:27:48.915228 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:48.915609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.916793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:48.916793 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:48.916793 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:48.916793 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:48.916793 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:48.916793 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:48.916793 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.916793 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:48.918442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.918442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:27:48.918442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:48.918442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:48.918442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:48.918442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:48.918442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:48.918442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:48.918442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:48.918442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:48.918442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:48.918442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:27:48.918442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:48.918442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:27:48.918442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:27:48.918930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:48.918930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:27:48.918930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:48.918930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:27:48.918930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:27:48.918930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:48.918930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:27:48.918930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.918930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:27:48.919723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.919723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:48.919723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:48.919723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:48.919723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.919723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:27:48.919723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:27:48.919723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:27:48.919723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:48.919723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:48.919723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:48.919723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:48.920302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:48.920302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:48.920302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:48.920302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:48.920302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:48.920302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:48.920302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:48.920302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:48.920302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:48.920302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:48.920302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:48.920302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:48.920302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:48.920302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:48.920935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:48.920935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:48.920935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.920935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:48.920935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.920935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:48.920935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:48.920935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:48.920935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:48.920935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:48.920935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.920935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:48.920935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:48.920935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.921554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:48.921554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:48.921554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.921554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:48.921554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:48.921554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.921554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:27:48.921554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:27:48.921554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:27:48.921554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:48.921554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:48.921554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:48.921554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:48.921554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:48.921554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:48.921554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:48.921554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:48.921554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:48.921554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:48.922444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:48.922444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:48.922444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:48.922444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:48.922444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:48.922444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:48.922444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:48.922444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:48.922444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:48.922444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:48.922444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.922444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:48.928257 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:48.928629 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:48.928785 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:48.929013 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:48.929607 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:48.930065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.930942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:48.930942 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:48.930942 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:48.930942 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:48.930942 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:48.930942 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:48.930942 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.930942 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:48.932243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.932243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:27:48.932243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:48.932243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:48.932243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:48.932243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:48.932243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:48.932243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:48.932243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:48.932243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:48.932243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:48.932243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:27:48.932243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:48.932243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:27:48.932243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:27:48.932656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:48.932656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:27:48.932656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:48.932656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:27:48.932656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:27:48.932656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:48.932656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:27:48.932656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.932656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:27:48.933762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.933762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:48.933762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:48.933762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:48.933762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.933762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:27:48.933762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:27:48.933762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:48.933762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:48.933762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:48.933762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:48.934112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:48.934112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:48.934112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:48.934112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:48.934112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:48.934112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:48.934112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:48.934112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:48.934112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:48.934112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:48.934112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:48.934112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:48.934112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:48.934112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:48.934502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:48.934502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:48.934502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.934502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:48.935576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.935576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:48.935576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:48.935576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:48.935576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:48.935576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:48.935576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.935576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:48.935576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:48.935576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.935576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:48.935576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:48.935576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.935576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:48.935576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:48.935576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.935576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:27:48.936483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:27:48.936483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:48.936483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:48.936483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:48.936483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:48.936483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:48.936483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:48.936483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:48.936483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:48.936483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:48.936483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:48.936483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:48.936483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:48.936483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:48.936483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:48.936483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:48.936483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:48.936483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:48.936483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:48.937394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:48.937394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:48.937394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.937394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:48.937394 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:48.937653 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:48.937904 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:48.938205 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:48.938761 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:48.939116 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:48.939352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.940170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:48.940170 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:48.940170 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:48.940170 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:48.940170 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:48.940170 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:48.940170 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.940170 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:48.941702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.941702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:48.941702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:48.941702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:48.941702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.941702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:27:48.941702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:48.941702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:48.941702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:48.941702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:48.941702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:48.941702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:48.942334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:48.942334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:48.942334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:48.942334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:48.942334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:48.942334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:48.942334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:48.942334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:48.942334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:48.942334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:48.942334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:48.942334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:48.942334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:48.942915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.942915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:48.944140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.944140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:48.944140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:48.944140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:48.944140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:48.944140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:48.944140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.944140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:48.944140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:48.944140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.944140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:48.944140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:48.944140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.944140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:48.944140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:48.944140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:48.944862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:27:48.944862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:48.944862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:48.944862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:48.944862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:48.944862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:48.944862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:48.944862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:48.944862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:48.944862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:48.944862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:48.944862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:48.944862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:48.944862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:48.944862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:48.944862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:48.944862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:48.944862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:48.944862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:48.945427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:48.945427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:48.945427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:53.624481 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:53.625255 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:53.625255 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:53.625963 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:53.658193 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:53.658634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.659528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:53.659528 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:53.659528 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:53.659528 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:53.659528 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:53.659528 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:53.659528 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.659528 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:53.660420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.660420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:53.660420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:53.660420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:53.660420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.660420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:27:53.660420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:53.660420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:53.660420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:53.660420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:53.660420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:53.660420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:53.660878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:53.660878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:53.660878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:53.660878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:53.660878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:53.660878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:53.660878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:53.660878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:53.660878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:53.660878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:53.660878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:53.660878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:53.660878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:53.661340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.661340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:53.661517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.661517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:53.661517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:53.661517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:53.661517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:53.661517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:53.661517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.661517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:53.661517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:53.661517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.661517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:53.661517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:53.661517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.661517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:53.661517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:53.661517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.661999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:27:53.661999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:53.661999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:53.661999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:53.661999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:53.661999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:53.661999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:53.661999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:53.661999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:53.661999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:53.661999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:53.661999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:53.661999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:53.661999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:53.661999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:53.661999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:53.661999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:53.661999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:53.661999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:53.663425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:53.663425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.663425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:53.859124 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:53.859705 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:53.859915 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:53.860969 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:53.861208 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:53.861356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.862265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:53.862265 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:53.862265 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:53.862265 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:53.862265 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:53.862265 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:53.862265 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.862265 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:53.863776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.863776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:53.863776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:53.863776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:53.863776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.863776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:27:53.863776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:53.863776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:53.863776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:53.863776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:53.863776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:53.863776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:53.864493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:53.864493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:53.864493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:53.864493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:53.864493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:53.864493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:53.864493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:53.864493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:53.864493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:53.864493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:53.864493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:53.864493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:53.864493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:53.865054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.865054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:53.865054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.865054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:53.865054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:53.865054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:53.865054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:53.865054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:53.865054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.865054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:53.865054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:53.865054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.865054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:53.865054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:53.865054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.865925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:53.865925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:53.865925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.865925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:27:53.865925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:53.865925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:53.865925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:53.865925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:53.865925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:53.865925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:53.865925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:53.865925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:53.865925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:53.865925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:53.865925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:53.865925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:53.865925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:53.865925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:53.865925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:53.866753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:53.866753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:53.866753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:53.866753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:53.866753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.866753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:53.913385 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:53.914093 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:53.914289 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:53.914494 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:27:53.918650 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:53.918650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.918650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:53.918650 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:53.918650 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:53.918650 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:53.918650 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:53.918650 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:53.918650 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.918650 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:53.918650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.918650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:27:53.918650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:53.918650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:53.919516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:53.919516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:53.919516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:53.919516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:53.919516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:53.919516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:53.919516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:53.919516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:27:53.919516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:53.919516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:27:53.919516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:27:53.919516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:53.919516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:27:53.919516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:53.919516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:27:53.919516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:27:53.919516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:53.919516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:27:53.921084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.921084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:27:53.921084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.921084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:53.921084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:53.921084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:53.921084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.921084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:27:53.921084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:27:53.921084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:27:53.921084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:53.921084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:53.922218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:53.922218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:53.922218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:53.922218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:53.922218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:53.922218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:53.922218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:53.922218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:53.922218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:53.922218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:53.922218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:53.922218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:53.922218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:53.922218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:53.923227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:53.923227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:53.923227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:53.923227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:53.923227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.923227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:53.923227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.923227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:53.923227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:53.923227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:53.923227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:53.923227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:53.923227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.923227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:53.923913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:53.923913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.923913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:53.923913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:53.923913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.923913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:53.923913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:53.923913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.923913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:27:53.923913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:27:53.923913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:27:53.923913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:53.923913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:53.923913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:53.923913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:53.923913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:53.923913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:53.923913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:53.923913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:53.924645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:53.924645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:53.924645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:53.924645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:53.924645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:53.924645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:53.924645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:53.924645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:53.924645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:53.924645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:53.924645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:53.924645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:53.924645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.924645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:53.928757 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:53.929085 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:53.929215 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:53.929346 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:53.929889 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:53.930259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.931183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:53.931183 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:53.931183 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:53.931183 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:53.931183 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:53.931183 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:53.931183 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.931183 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:53.932360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.932360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:27:53.932360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:53.932360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:53.932360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:53.932360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:53.932360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:53.932360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:53.932360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:53.932360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:53.932360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:53.932360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:27:53.932360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:53.932360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:27:53.932360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:27:53.932899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:53.932899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:27:53.932899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:53.932899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:27:53.932899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:27:53.932899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:53.932899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:27:53.932899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.932899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:27:53.933755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.933755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:53.933755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:53.933755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:53.933755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.933755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:27:53.933755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:27:53.933755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:53.933755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:53.933755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:53.933755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:53.934147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:53.934147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:53.934147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:53.934147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:53.934147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:53.934147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:53.934147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:53.934147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:53.934147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:53.934147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:53.934147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:53.934147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:53.934147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:53.934147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:53.934524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:53.934524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:53.934524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.934524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:53.935912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.935912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:53.935912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:53.935912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:53.935912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:53.935912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:53.935912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.935912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:53.935912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:53.935912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.935912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:53.935912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:53.935912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.935912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:53.935912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:53.935912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.935912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:27:53.936942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:27:53.936942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:53.936942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:53.936942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:53.936942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:53.936942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:53.936942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:53.936942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:53.936942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:53.936942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:53.936942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:53.936942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:53.936942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:53.936942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:53.936942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:53.936942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:53.936942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:53.936942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:53.936942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:53.937790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:53.937790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:53.937790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.937790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:53.938489 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:53.938990 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:53.939194 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:53.939413 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:53.939886 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:53.940228 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:53.940456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.941120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:53.941120 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:53.941120 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:53.941120 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:53.941120 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:53.941120 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:53.941120 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.941120 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:53.942045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.942045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:53.942045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:53.942045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:53.942045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.942045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:27:53.942045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:53.942045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:53.942045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:53.942045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:53.942045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:53.942045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:53.942486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:53.942486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:53.942486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:53.942486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:53.942486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:53.942486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:53.942486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:53.942486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:53.942486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:53.942486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:53.942486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:53.942486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:53.942486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:53.942846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.942846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:53.943016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.943016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:53.943016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:53.943016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:53.943016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:53.943016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:53.943016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.943016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:53.943016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:53.943016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.943016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:53.943016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:53.943016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.943016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:53.943016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:53.943016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:53.943479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:27:53.943479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:53.943479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:53.943479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:53.943479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:53.943479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:53.943479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:53.943479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:53.943479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:53.943479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:53.943479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:53.943479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:53.943479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:53.943479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:53.943479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:53.943479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:53.943479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:53.943479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:53.943479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:53.944035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:53.944035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:53.944035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:58.625094 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:58.625638 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:58.625638 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:58.626696 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:58.662074 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:58.662302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.662950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:58.662950 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:58.662950 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:58.662950 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:58.662950 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:58.662950 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:58.662950 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.662950 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:58.664700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.664700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:58.664700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:58.664700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:58.664700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.664700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:27:58.664700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:58.664700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:58.664700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:58.664700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:58.664700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:58.664700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:58.665404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:58.665404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:58.665404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:58.665404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:58.665404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:58.665404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:58.665404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:58.665404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:58.665404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:58.665404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:58.665404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:58.665404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:58.665404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:58.666027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.666027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:58.666463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.666463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:58.666463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:58.666463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:58.666463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:58.666463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:58.666463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.666463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:58.666463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:58.666463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.666463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:58.666463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:58.666463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.666463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:58.666463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:58.666463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.667235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:27:58.667235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:58.667235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:58.667235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:58.667235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:58.667235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:58.667235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:58.667235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:58.667235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:58.667235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:58.667235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:58.667235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:58.667235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:58.667235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:58.667235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:58.667235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:58.667235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:58.667235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:58.667235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:58.668250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:58.668250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.668250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:58.859818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:58.860375 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:58.860580 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:58.861580 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:58.861984 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:58.862329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.863212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:58.863212 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:58.863212 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:58.863212 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:58.863212 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:58.863212 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:58.863212 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.863212 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:58.865239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.865239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:58.865239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:58.865239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:58.865239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.865239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:27:58.865239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:58.865239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:58.865239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:58.865239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:58.865239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:58.865239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:58.865870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:58.865870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:58.865870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:58.865870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:58.865870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:58.865870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:58.865870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:58.865870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:58.865870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:58.865870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:58.865870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:58.865870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:58.865870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:58.866417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.866417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:58.868440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.868440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:58.868440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:58.868440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:58.868440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:58.868440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:58.868440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.868440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:58.868440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:58.868440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.868440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:58.868440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:58.868440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.868440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:58.868440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:58.868440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.869410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:27:58.869410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:58.869410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:58.869410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:58.869410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:58.869410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:58.869410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:58.869410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:58.869410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:58.869410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:58.869410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:58.869410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:58.869410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:58.869410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:58.869410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:58.869410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:58.869410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:58.869410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:58.869410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:58.870292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:58.870292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.870292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:58.913966 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:58.914237 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:58.914402 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:58.914557 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:27:58.915106 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:58.915449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.916170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:58.916170 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:58.916170 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:58.916170 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:58.916170 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:58.916170 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:58.916170 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.916170 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:58.917265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.917265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:27:58.917265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:58.917265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:58.917265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:58.917265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:58.917265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:58.917265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:58.917265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:58.917265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:58.917265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:58.917265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:27:58.917265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:58.917265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:27:58.917265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:27:58.917956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:58.917956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:27:58.917956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:58.917956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:27:58.917956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:27:58.917956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:58.917956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:27:58.917956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.917956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:27:58.918668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.918668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:58.918668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:58.918668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:58.918668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.918668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:27:58.918668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:27:58.918668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:27:58.918668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:58.918668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:58.918668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:58.918668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:58.919204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:58.919204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:58.919204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:58.919204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:58.919204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:58.919204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:58.919204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:58.919204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:58.919204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:58.919204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:58.919204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:58.919204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:58.919204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:58.919204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:58.919774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:58.919774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:58.919774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.919774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:58.919774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.919774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:58.919774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:58.919774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:58.919774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:58.919774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:58.919774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.919774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:58.919774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:58.919774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.920370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:58.920370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:58.920370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.920370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:58.920370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:58.920370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.920370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:27:58.920370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:27:58.920370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:27:58.920370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:58.920370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:58.920370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:58.920370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:58.920370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:58.920370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:58.920370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:58.920370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:58.920370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:58.920370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:58.921152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:58.921152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:58.921152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:58.921152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:58.921152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:58.921152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:58.921152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:58.921152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:58.921152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:58.921152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:58.921152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.921152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:58.929209 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:58.929576 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:58.929813 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:58.930037 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:58.930506 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:58.931070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.932559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:58.932559 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:58.932559 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:58.932559 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:58.932559 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:58.932559 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:58.932559 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.932559 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:58.933977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.933977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:27:58.933977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:58.933977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:58.933977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:58.933977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:58.933977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:58.933977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:58.933977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:58.933977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:27:58.933977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:58.933977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:27:58.933977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:58.933977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:27:58.933977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:27:58.934584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:58.934584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:27:58.934584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:58.934584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:27:58.934584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:27:58.934584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:58.934584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:27:58.934584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.934584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:27:58.935465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.935465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:58.935465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:58.935465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:58.935465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.935465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:27:58.935465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:27:58.935465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:58.935465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:58.935465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:58.935465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:58.936241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:58.936241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:58.936241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:58.936241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:58.936241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:58.936241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:58.936241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:58.936241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:58.936241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:58.936241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:58.936241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:58.936241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:58.936241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:58.936241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:58.937065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:58.937065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:58.937065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.937065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:58.937365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.937365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:58.937365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:58.937365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:58.937365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:58.937365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:58.937365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.937365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:58.937365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:58.937365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.937365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:58.937365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:58.937365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.937365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:58.937365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:58.937365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.937365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:27:58.938354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:27:58.938354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:27:58.938354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:27:58.938354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:58.938354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:58.938354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:58.938354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:58.938354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:58.938354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:58.938354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:58.938354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:58.938354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:58.938354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:58.938354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:58.938354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:58.938354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:58.938354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:58.938354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:58.938354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:58.939551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:58.939551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:58.939551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.939551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:27:58.939551 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:27:58.939551 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:27:58.939551 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:27:58.939551 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:27:58.940127 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:58.940127 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:27:58.940286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.940703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:27:58.940703 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:58.940703 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:27:58.940703 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:27:58.940703 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:58.940703 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:27:58.940703 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.940703 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:27:58.941697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.941697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:27:58.941697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:58.941697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:27:58.941697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.941697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:27:58.941697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:58.941697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:58.941697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:58.941697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:58.941697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:58.941697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:58.942401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:27:58.942401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:58.942401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:27:58.942401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:58.942401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:27:58.942401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:27:58.942401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:58.942401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:27:58.942401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:58.942401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:27:58.942401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:27:58.942401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:58.942401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:27:58.942772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.942772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:27:58.942772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.942772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:27:58.942772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:27:58.942772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:27:58.942772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:27:58.942772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:27:58.942772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.942772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:27:58.942772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:27:58.942772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.942772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:27:58.942772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:27:58.942772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.943136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:27:58.943136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:27:58.943136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:27:58.943136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:27:58.943136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:27:58.943136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:58.943136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:58.943136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:27:58.943136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:27:58.943136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:27:58.943136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:27:58.943136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:27:58.943136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:27:58.943136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:27:58.943136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:27:58.943136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:27:58.943136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:27:58.943136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:27:58.943136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:27:58.943594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:27:58.943594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:27:58.943594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:27:58.943594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:27:58.943594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:27:58.943594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:03.625816 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:03.626967 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:03.626967 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:03.627255 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:03.633137 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=240 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:28:03.662126 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:03.662440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.663159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:03.663159 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:03.663159 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:03.663159 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:03.663159 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:03.663159 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:03.663159 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.663159 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:03.663837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.663837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:03.663837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:03.663837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:03.663837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.663837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:28:03.663837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:03.663837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:03.663837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:03.663837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:03.663837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:03.663837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:03.664520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:03.664520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:03.664520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:03.664520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:03.664520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:03.664520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:03.664520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:03.664520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:03.664520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:03.664520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:03.664520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:03.664520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:03.664520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:03.665410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.665410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:03.665410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.665410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:03.665410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:03.665410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:03.665410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:03.665410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:03.665410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.665410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:03.665410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:03.665410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.665410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:03.665410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:03.665410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.666375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:03.666375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:03.666375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.666375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:28:03.666375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:03.666375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:03.666375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:03.666375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:03.666375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:03.666375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:03.666375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:03.666375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:03.666375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:03.666375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:03.666375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:03.666375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:03.666375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:03.666375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:03.666375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:03.667349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:03.667349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:03.667349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:03.667349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:03.667349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.667349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:03.860559 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:03.861049 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:03.861256 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:03.862184 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:03.862475 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:03.862745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.863559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:03.863559 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:03.863559 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:03.863559 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:03.863559 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:03.863559 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:03.863559 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.863559 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:03.865399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.865399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:03.865399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:03.865399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:03.865399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.865399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:28:03.865399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:03.865399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:03.865399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:03.865399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:03.865399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:03.865399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:03.866056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:03.866056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:03.866056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:03.866056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:03.866056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:03.866056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:03.866056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:03.866056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:03.866056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:03.866056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:03.866056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:03.866056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:03.866056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:03.866708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.866708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:03.867511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.867511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:03.867511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:03.867511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:03.867511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:03.867511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:03.867511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.867511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:03.867511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:03.867511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.867511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:03.867511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:03.867511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.867511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:03.867511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:03.867511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.868406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:28:03.868406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:03.868406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:03.868406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:03.868406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:03.868406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:03.868406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:03.868406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:03.868406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:03.868406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:03.868406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:03.868406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:03.868406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:03.868406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:03.868406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:03.868406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:03.868406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:03.868406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:03.868406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:03.869333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:03.869333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.869333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:03.914575 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:03.915021 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:03.915289 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:03.915801 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:28:03.916535 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:03.917029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.917807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:03.917807 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:03.917807 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:03.917807 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:03.917807 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:03.917807 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:03.917807 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.917807 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:03.918499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.918499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:28:03.918499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:03.918499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:03.918499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:03.918499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:03.918499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:03.918499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:03.918499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:03.918499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:03.918499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:03.918499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:28:03.918499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:03.918499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:28:03.918499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:28:03.918998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:03.918998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:28:03.918998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:03.918998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:28:03.918998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:28:03.918998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:03.918998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:28:03.918998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.918998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:28:03.919500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.919500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:03.919500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:03.919500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:03.919500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.919500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:28:03.919500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:28:03.919500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:28:03.919500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:03.919500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:03.919500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:03.919500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:03.919895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:03.919895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:03.919895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:03.919895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:03.919895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:03.919895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:03.919895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:03.919895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:03.919895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:03.919895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:03.919895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:03.919895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:03.919895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:03.919895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:03.920292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:03.920292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:03.920292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.920292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:03.920428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.920428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:03.920428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:03.920428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:03.920428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:03.920428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:03.920428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.920428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:03.920428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:03.920428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.920428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:03.920428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:03.920428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.920428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:03.920428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:03.920428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.920851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:28:03.920851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:28:03.920851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:28:03.920851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:03.920851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:03.920851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:03.920851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:03.920851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:03.920851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:03.920851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:03.920851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:03.920851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:03.920851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:03.920851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:03.920851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:03.920851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:03.920851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:03.920851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:03.921324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:03.921324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:03.921324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:03.921324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:03.921324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:03.921324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.921324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:03.929657 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:03.929955 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:03.930078 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:03.930202 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:03.930600 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:03.930996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.931699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:03.931699 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:03.931699 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:03.931699 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:03.931699 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:03.931699 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:03.931699 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.931699 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:03.932509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.932509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:28:03.932509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:03.932509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:03.932509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:03.932509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:03.932509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:03.932509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:03.932509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:03.932509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:03.932509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:03.932509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:28:03.932509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:03.932509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:28:03.932509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:28:03.933208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:03.933208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:28:03.933208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:03.933208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:28:03.933208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:28:03.933208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:03.933208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:28:03.933208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.933208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:28:03.933607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.933607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:03.933607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:03.933607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:03.933607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.933607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:28:03.933607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:28:03.933607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:03.933607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:03.933607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:03.933607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:03.934117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:03.934117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:03.934117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:03.934117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:03.934117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:03.934117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:03.934117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:03.934117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:03.934117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:03.934117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:03.934117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:03.934117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:03.934117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:03.934117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:03.934672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:03.934672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:03.934672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.934672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:03.934672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.934672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:03.934672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:03.934672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:03.934672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:03.934672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:03.934672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.934672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:03.934672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:03.934672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.935282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:03.935282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:03.935282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.935282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:03.935282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:03.935282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.935282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:28:03.935282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:28:03.935282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:03.935282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:03.935282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:03.935282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:03.935282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:03.935282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:03.935282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:03.935282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:03.935282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:03.935282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:03.935282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:03.936373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:03.936373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:03.936373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:03.936373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:03.936373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:03.936373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:03.936373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:03.936373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:03.936373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:03.936373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.936373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:03.939254 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:03.939543 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:03.939656 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:03.939826 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:03.940170 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:03.940515 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:03.940789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.941434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:03.941434 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:03.941434 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:03.941434 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:03.941434 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:03.941434 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:03.941434 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.941434 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:03.942263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.942263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:03.942263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:03.942263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:03.942263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.942263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:28:03.942263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:03.942263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:03.942263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:03.942263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:03.942263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:03.942263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:03.942607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:03.942607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:03.942607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:03.942607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:03.942607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:03.942607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:03.942607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:03.942607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:03.942607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:03.942607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:03.942607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:03.942607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:03.942607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:03.942981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.942981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:03.943232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.943232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:03.943232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:03.943232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:03.943232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:03.943232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:03.943232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.943232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:03.943232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:03.943232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.943232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:03.943232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:03.943232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.943232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:03.943232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:03.943232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:03.943747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:28:03.943747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:03.943747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:03.943747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:03.943747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:03.943747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:03.943747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:03.943747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:03.943747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:03.943747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:03.943747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:03.943747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:03.943747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:03.943747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:03.943747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:03.943747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:03.943747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:03.943747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:03.943747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:03.944281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:03.944281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:03.944281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:08.626637 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:08.627452 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:08.627635 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:08.629014 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:08.664593 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:08.665181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.666381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:08.666381 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:08.666381 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:08.666381 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:08.666381 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:08.666381 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:08.666381 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.666381 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:08.667447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.667447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:08.667447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:08.667447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:08.667447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.667447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:28:08.667447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:08.667447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:08.667447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:08.667447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:08.667447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:08.667447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:08.667854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:08.667854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:08.667854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:08.667854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:08.667854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:08.667854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:08.667854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:08.667854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:08.667854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:08.667854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:08.667854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:08.667854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:08.667854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:08.668244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.668244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:08.668506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.668506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:08.668506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:08.668506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:08.668506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:08.668506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:08.668506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.668506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:08.668506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:08.668506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.668506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:08.668506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:08.668506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.668506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:08.668506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:08.668506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.669034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:28:08.669034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:08.669034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:08.669034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:08.669034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:08.669034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:08.669034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:08.669034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:08.669034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:08.669034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:08.669034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:08.669034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:08.669034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:08.669034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:08.669034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:08.669034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:08.669034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:08.669034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:08.669034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:08.669540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:08.669540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.669540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:08.861143 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:08.861772 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:08.861984 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:08.863195 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:08.863574 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:08.863890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.865299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:08.865299 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:08.865299 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:08.865299 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:08.865299 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:08.865299 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:08.865299 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.865299 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:08.866951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.866951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:08.866951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:08.866951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:08.866951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.866951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:28:08.866951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:08.866951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:08.866951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:08.866951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:08.866951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:08.866951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:08.867499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:08.867499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:08.867499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:08.867499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:08.867499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:08.867499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:08.867499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:08.867499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:08.867499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:08.867499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:08.867499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:08.867499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:08.867499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:08.868097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.868097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:08.868903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.868903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:08.868903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:08.868903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:08.868903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:08.868903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:08.868903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.868903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:08.868903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:08.868903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.868903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:08.868903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:08.868903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.868903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:08.868903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:08.868903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.869627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:28:08.869627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:08.869627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:08.869627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:08.869627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:08.869627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:08.869627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:08.869627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:08.869627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:08.869627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:08.869627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:08.869627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:08.869627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:08.869627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:08.869627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:08.869627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:08.869627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:08.869627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:08.869627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:08.870445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:08.870445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.870445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:08.915194 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:08.915914 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:08.916170 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:08.916499 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:28:08.917231 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:08.917652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.918666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:08.918666 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:08.918666 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:08.918666 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:08.918666 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:08.918666 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:08.918666 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.918666 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:08.920216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.920216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:28:08.920216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:08.920216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:08.920216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:08.920216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:08.920216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:08.920216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:08.920216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:08.920216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:08.920216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:08.920216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:28:08.920216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:08.920216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:28:08.920216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:28:08.921166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:08.921166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:28:08.921166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:08.921166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:28:08.921166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:28:08.921166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:08.921166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:28:08.921166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.921166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:28:08.922128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.922128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:08.922128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:08.922128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:08.922128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.922128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:28:08.922128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:28:08.922128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:28:08.922128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:08.922128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:08.922128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:08.922128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:08.923137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:08.923137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:08.923137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:08.923137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:08.923137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:08.923137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:08.923137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:08.923137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:08.923137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:08.923137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:08.923137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:08.923137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:08.923137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:08.923137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:08.924030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:08.924030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:08.924030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.924030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:08.926260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.926260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:08.926260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:08.926260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:08.926260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:08.926260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:08.926260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.926260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:08.926260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:08.926260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.926260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:08.926260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:08.926260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.926260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:08.926260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:08.926260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.926834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:28:08.926834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:28:08.926834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:28:08.926834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:08.926834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:08.926834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:08.926834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:08.926834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:08.926834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:08.926834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:08.926834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:08.926834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:08.926834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:08.926834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:08.926834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:08.926834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:08.926834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:08.926834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:08.927345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:08.927345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:08.927345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:08.927345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:08.927345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:08.927345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.927345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:08.930241 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:08.930586 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:08.930765 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:08.930967 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:08.931514 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:08.931964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.932817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:08.932817 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:08.932817 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:08.932817 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:08.932817 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:08.932817 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:08.932817 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.932817 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:08.933765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.933765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:28:08.933765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:08.933765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:08.933765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:08.933765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:08.933765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:08.933765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:08.933765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:08.933765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:08.933765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:08.933765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:28:08.933765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:08.933765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:28:08.933765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:28:08.934403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:08.934403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:28:08.934403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:08.934403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:28:08.934403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:28:08.934403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:08.934403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:28:08.934403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.934403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:28:08.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:08.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:08.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:08.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:28:08.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:28:08.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:08.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:08.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:08.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:08.935531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:08.935531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:08.935531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:08.935531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:08.935531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:08.935531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:08.935531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:08.935531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:08.935531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:08.935531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:08.935531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:08.935531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:08.935531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:08.935531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:08.936185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:08.936185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:08.936185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.936185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:08.936637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.936637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:08.936637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:08.936637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:08.936637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:08.936637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:08.936637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.936637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:08.936637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:08.936637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.936637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:08.936637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:08.936637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.936637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:08.936637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:08.936637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.936637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:28:08.937208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:28:08.937208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:08.937208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:08.937208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:08.937208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:08.937208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:08.937208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:08.937208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:08.937208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:08.937208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:08.937208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:08.937208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:08.937208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:08.937208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:08.937208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:08.937208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:08.937208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:08.937208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:08.937208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:08.937746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:08.937746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:08.937746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.937746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:08.939777 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:08.940125 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:08.940287 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:08.940500 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:08.941019 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:08.941398 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:08.941793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.942822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:08.942822 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:08.942822 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:08.942822 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:08.942822 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:08.942822 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:08.942822 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.942822 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:08.943826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.943826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:08.943826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:08.943826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:08.943826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.943826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:28:08.943826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:08.943826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:08.943826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:08.943826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:08.943826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:08.943826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:08.944393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:08.944393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:08.944393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:08.944393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:08.944393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:08.944393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:08.944393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:08.944393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:08.944393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:08.944393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:08.944393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:08.944393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:08.944393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:08.945010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.945010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:08.945298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.945298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:08.945298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:08.945298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:08.945298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:08.945298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:08.945298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.945298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:08.945298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:08.945298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.945298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:08.945298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:08.945298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.945298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:08.945298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:08.945298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:08.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:28:08.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:08.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:08.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:08.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:08.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:08.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:08.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:08.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:08.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:08.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:08.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:08.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:08.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:08.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:08.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:08.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:08.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:08.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:08.946357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:08.946357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:08.946357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:13.627384 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:13.628208 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:13.628208 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:13.629436 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:13.672780 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:13.672954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.673583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:13.673583 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:13.673583 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:13.673583 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:13.673583 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:13.673583 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:13.673583 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.673583 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:13.674609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.674609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:13.674609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:13.674609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:13.674609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.674609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:28:13.674609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:13.674609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:13.674609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:13.674609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:13.674609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:13.674609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:13.675369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:13.675369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:13.675369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:13.675369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:13.675369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:13.675369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:13.675369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:13.675369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:13.675369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:13.675369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:13.675369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:13.675369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:13.675369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:13.675935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.675935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:13.675935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.675935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:13.675935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:13.675935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:13.675935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:13.675935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:13.675935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.675935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:13.675935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:13.675935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.675935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:13.675935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:13.675935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.676655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:13.676655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:13.676655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.676655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:28:13.676655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:13.676655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:13.676655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:13.676655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:13.676655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:13.676655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:13.676655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:13.676655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:13.676655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:13.676655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:13.676655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:13.676655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:13.676655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:13.676655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:13.676655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:13.677534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:13.677534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:13.677534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:13.677534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:13.677534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.677534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:13.861901 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:13.862327 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:13.862573 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:13.863611 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:13.863892 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:13.864176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.864990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:13.864990 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:13.864990 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:13.864990 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:13.864990 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:13.864990 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:13.864990 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.864990 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:13.866840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.866840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:13.866840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:13.866840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:13.866840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.866840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:28:13.866840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:13.866840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:13.866840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:13.866840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:13.866840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:13.866840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:13.867939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:13.867939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:13.867939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:13.867939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:13.867939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:13.867939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:13.867939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:13.867939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:13.867939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:13.867939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:13.867939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:13.867939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:13.867939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:13.868950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.868950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:13.869843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.869843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:13.869843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:13.869843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:13.869843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:13.869843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:13.869843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.869843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:13.869843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:13.869843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.869843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:13.869843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:13.869843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.869843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:13.869843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:13.869843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.870375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:28:13.870375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:13.870375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:13.870375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:13.870375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:13.870375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:13.870375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:13.870375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:13.870375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:13.870375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:13.870375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:13.870375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:13.870375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:13.870375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:13.870375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:13.870375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:13.870375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:13.870375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:13.870375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:13.870931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:13.870931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.870931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:13.916113 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:13.916382 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:13.916545 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:13.916753 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:28:13.917221 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:13.917503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.918218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:13.918218 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:13.918218 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:13.918218 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:13.918218 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:13.918218 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:13.918218 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.918218 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:13.918870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.918870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:28:13.918870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:13.918870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:13.918870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:13.918870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:13.918870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:13.918870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:13.918870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:13.918870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:13.918870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:13.918870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:28:13.918870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:13.918870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:28:13.918870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:28:13.919396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:13.919396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:28:13.919396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:13.919396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:28:13.919396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:28:13.919396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:13.919396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:28:13.919396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.919396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:28:13.919763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.919763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:13.919763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:13.919763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:13.919763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.919763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:28:13.919763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:28:13.919763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:28:13.919763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:13.919763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:13.919763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:13.919763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:13.920139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:13.920139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:13.920139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:13.920139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:13.920139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:13.920139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:13.920139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:13.920139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:13.920139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:13.920139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:13.920139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:13.920139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:13.920139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:13.920139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:13.920549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:13.920549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:13.920549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.920549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:13.920772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.920772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:13.920772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:13.920772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:13.920772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:13.920772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:13.920772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.920772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:13.920772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:13.920772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.920772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:13.920772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:13.920772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.920772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:13.920772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:13.920772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.921118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:28:13.921118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:28:13.921118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:28:13.921118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:13.921118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:13.921118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:13.921118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:13.921118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:13.921118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:13.921118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:13.921118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:13.921118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:13.921118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:13.921118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:13.921118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:13.921118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:13.921118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:13.921118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:13.921471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:13.921471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:13.921471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:13.921471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:13.921471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:13.921471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.921471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:13.930779 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:13.931071 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:13.931253 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:13.931445 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:13.931896 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:13.932270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.933056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:13.933056 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:13.933056 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:13.933056 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:13.933056 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:13.933056 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:13.933056 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.933056 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:13.933800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.933800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:28:13.933800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:13.933800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:13.933800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:13.933800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:13.933800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:13.933800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:13.933800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:13.933800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:13.933800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:13.933800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:28:13.933800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:13.933800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:28:13.933800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:28:13.934776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:13.934776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:28:13.934776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:13.934776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:28:13.934776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:28:13.934776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:13.934776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:28:13.934776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.934776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:28:13.934776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.934776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:13.934776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:13.934776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:13.934776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.935655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:28:13.935655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:28:13.935655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:13.935655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:13.935655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:13.935655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:13.935655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:13.935655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:13.935655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:13.935655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:13.935655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:13.935655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:13.935655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:13.936321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:13.936321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:13.936321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:13.936321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:13.936321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:13.936321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:13.936321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:13.936321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:13.936321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:13.936321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.936321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:13.936321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.937093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:13.937093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:13.937093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:13.937093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:13.937093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:13.937093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.937093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:13.937093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:13.937093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.937093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:13.937093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:13.937093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.937093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:13.937093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:13.937093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.937093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:28:13.937093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:28:13.937093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:13.937093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:13.938099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:13.938099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:13.938099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:13.938099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:13.938099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:13.938099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:13.938099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:13.938099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:13.938099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:13.938099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:13.938099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:13.938099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:13.938099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:13.938099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:13.938099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:13.938099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:13.938099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:13.938099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:13.938099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.938099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:13.940138 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:13.940582 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:13.940727 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:13.940820 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:13.941075 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:13.941275 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:13.941426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.941894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:13.941894 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:13.941894 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:13.941894 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:13.941894 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:13.941894 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:13.941894 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.941894 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:13.942696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.942696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:13.942696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:13.942696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:13.942696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.942696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:28:13.942696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:13.942696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:13.942696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:13.942696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:13.942696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:13.942696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:13.943084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:13.943084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:13.943084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:13.943084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:13.943084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:13.943084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:13.943084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:13.943084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:13.943084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:13.943084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:13.943084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:13.943084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:13.943084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:13.943490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.943490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:13.943702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.943702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:13.943702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:13.943702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:13.943702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:13.943702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:13.943702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.943702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:13.943702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:13.943702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.943702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:13.943702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:13.943702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.943702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:13.943702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:13.943702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:13.944164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:28:13.944164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:13.944164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:13.944164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:13.944164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:13.944164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:13.944164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:13.944164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:13.944164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:13.944164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:13.944164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:13.944164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:13.944164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:13.944164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:13.944164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:13.944164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:13.944164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:13.944164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:13.944164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:13.944721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:13.944721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:13.944721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:18.628059 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:18.628626 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:18.628626 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:18.629430 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:18.662233 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:18.662412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.663187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:18.663187 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:18.663187 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:18.663187 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:18.663187 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:18.663187 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:18.663187 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.663187 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:18.664254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.664254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:18.664254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:18.664254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:18.664254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.664254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:28:18.664254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:18.664254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:18.664254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:18.664254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:18.664254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:18.664254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:18.665128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:18.665128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:18.665128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:18.665128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:18.665128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:18.665128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:18.665128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:18.665128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:18.665128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:18.665128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:18.665128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:18.665128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:18.665128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:18.665925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.665925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:18.665925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.665925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:18.665925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:18.665925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:18.665925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:18.665925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:18.665925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.665925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:18.665925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:18.665925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.665925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:18.665925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:18.665925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.666843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:18.666843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:18.666843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.666843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:28:18.666843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:18.666843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:18.666843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:18.666843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:18.666843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:18.666843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:18.666843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:18.666843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:18.666843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:18.666843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:18.666843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:18.666843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:18.666843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:18.666843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:18.666843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:18.667996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:18.667996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:18.667996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:18.667996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:18.667996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.667996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:18.862611 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:18.863395 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:18.863600 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:18.864670 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:18.865036 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:18.865273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.866486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:18.866486 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:18.866486 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:18.866486 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:18.866486 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:18.866486 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:18.866486 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.866486 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:18.868229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.868229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:18.868229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:18.868229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:18.868229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.868229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:28:18.868229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:18.868229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:18.868229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:18.868229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:18.868229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:18.868229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:18.868830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:18.868830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:18.868830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:18.868830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:18.868830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:18.868830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:18.868830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:18.868830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:18.868830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:18.868830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:18.868830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:18.868830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:18.868830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:18.869362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.869362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:18.870005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.870005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:18.870005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:18.870005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:18.870005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:18.870005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:18.870005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.870005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:18.870005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:18.870005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.870005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:18.870005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:18.870005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.870005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:18.870005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:18.870005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.870750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:28:18.870750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:18.870750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:18.870750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:18.870750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:18.870750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:18.870750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:18.870750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:18.870750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:18.870750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:18.870750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:18.870750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:18.870750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:18.870750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:18.870750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:18.870750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:18.870750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:18.870750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:18.870750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:18.871531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:18.871531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.871531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:18.916626 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:18.917315 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:18.917603 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:18.917841 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:28:18.918419 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:18.918784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.919779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:18.919779 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:18.919779 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:18.919779 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:18.919779 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:18.919779 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:18.919779 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.919779 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:18.921101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.921101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:28:18.921101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:18.921101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:18.921101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:18.921101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:18.921101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:18.921101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:18.921101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:18.921101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:18.921101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:18.921101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:28:18.921101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:18.921101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:28:18.921101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:28:18.921917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:18.921917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:28:18.921917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:18.921917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:28:18.921917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:28:18.921917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:18.921917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:28:18.921917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.921917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:28:18.923141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.923141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:18.923141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:18.923141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:18.923141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.923141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:28:18.923141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:28:18.923141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:28:18.923141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:18.923141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:18.923141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:18.923141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:18.923951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:18.923951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:18.923951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:18.923951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:18.923951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:18.923951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:18.923951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:18.923951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:18.923951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:18.923951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:18.923951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:18.923951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:18.923951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:18.923951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:18.924715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:18.924715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:18.924715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.924715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:18.924715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.924715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:18.924715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:18.924715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:18.924715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:18.924715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:18.924715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.924715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:18.924715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:18.924715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.925903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:18.925903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:18.925903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.925903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:18.925903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:18.925903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.925903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:28:18.925903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:28:18.925903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:28:18.925903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:18.925903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:18.925903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:18.925903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:18.925903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:18.925903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:18.925903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:18.925903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:18.925903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:18.925903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:18.926917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:18.926917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:18.926917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:18.926917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:18.926917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:18.926917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:18.926917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:18.926917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:18.926917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:18.926917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:18.926917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.926917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:18.931431 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:18.931936 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:18.932295 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:18.932515 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:18.933102 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:18.933568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.934415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:18.934415 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:18.934415 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:18.934415 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:18.934415 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:18.934415 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:18.934415 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.934415 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:18.935635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.935635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:28:18.935635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:18.935635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:18.935635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:18.935635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:18.935635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:18.935635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:18.935635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:18.935635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:18.935635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:18.935635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:28:18.935635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:18.935635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:28:18.935635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:28:18.936082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:18.936082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:28:18.936082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:18.936082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:28:18.936082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:28:18.936082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:18.936082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:28:18.936082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.936082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:28:18.937083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.937083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:18.937083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:18.937083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:18.937083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.937083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:28:18.937083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:28:18.937083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:18.937083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:18.937083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:18.937083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:18.937450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:18.937450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:18.937450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:18.937450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:18.937450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:18.937450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:18.937450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:18.937450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:18.937450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:18.937450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:18.937450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:18.937450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:18.937450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:18.937450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:18.937805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:18.937805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:18.937805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.937805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:18.939797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.939797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:18.939797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:18.939797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:18.939797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:18.939797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:18.939797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.939797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:18.939797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:18.939797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.939797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:18.939797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:18.939797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.939797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:18.939797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:18.939797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.939797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:28:18.940298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:28:18.940298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:18.940298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:18.940298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:18.940298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:18.940298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:18.940298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:18.940298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:18.940298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:18.940298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:18.940298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:18.940298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:18.940298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:18.940298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:18.940298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:18.940298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:18.940298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:18.940298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:18.940298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:18.940828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:18.940828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:18.940828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.940828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:18.940828 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:18.941167 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:18.941337 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:18.941493 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:18.941843 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:18.942041 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:18.942180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.942624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:18.942624 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:18.942624 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:18.942624 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:18.942624 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:18.942624 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:18.942624 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.942624 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:18.943249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.943249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:18.943249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:18.943249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:18.943249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.943249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:28:18.943249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:18.943249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:18.943249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:18.943249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:18.943249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:18.943249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:18.943897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:18.943897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:18.943897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:18.943897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:18.943897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:18.943897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:18.943897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:18.943897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:18.943897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:18.943897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:18.943897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:18.943897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:18.943897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:18.944584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.944584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:18.944584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.944584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:18.944584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:18.944584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:18.944584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:18.944584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:18.944584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.944584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:18.944584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:18.944584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.944584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:18.944584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:18.944584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.945392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:18.945392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:18.945392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:18.945392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:28:18.945392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:18.945392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:18.945392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:18.945392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:18.945392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:18.945392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:18.945392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:18.945392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:18.945392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:18.945392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:18.945392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:18.945392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:18.945392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:18.945392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:18.945392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:18.946457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:18.946457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:18.946457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:18.946457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:18.946457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:18.946457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:23.629290 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:23.630275 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:23.630275 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:23.631479 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:23.666890 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:23.667080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.667891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:23.667891 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:23.667891 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:23.667891 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:23.667891 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:23.667891 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:23.667891 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.667891 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:23.668630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.668630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:23.668630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:23.668630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:23.668630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.668630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:28:23.668630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:23.668630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:23.668630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:23.668630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:23.668630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:23.668630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:23.669345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:23.669345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:23.669345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:23.669345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:23.669345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:23.669345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:23.669345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:23.669345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:23.669345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:23.669345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:23.669345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:23.669345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:23.669345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:23.669971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.669971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:23.669971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.669971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:23.669971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:23.669971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:23.669971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:23.669971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:23.669971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.669971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:23.669971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:23.669971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.669971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:23.669971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:23.669971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.670749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:23.670749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:23.670749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.670749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:28:23.670749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:23.670749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:23.670749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:23.670749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:23.670749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:23.670749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:23.670749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:23.670749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:23.670749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:23.670749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:23.670749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:23.670749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:23.670749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:23.670749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:23.670749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:23.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:23.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:23.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:23.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:23.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:23.863471 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:23.863885 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:23.864119 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:23.865115 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:23.865403 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:23.865648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.867438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:23.867438 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:23.867438 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:23.867438 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:23.867438 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:23.867438 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:23.867438 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.867438 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:23.868315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.868315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:23.868315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:23.868315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:23.868315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.868315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:28:23.868315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:23.868315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:23.868315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:23.868315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:23.868315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:23.868315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:23.869124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:23.869124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:23.869124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:23.869124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:23.869124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:23.869124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:23.869124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:23.869124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:23.869124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:23.869124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:23.869124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:23.869124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:23.869124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:23.870624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.870624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:23.870624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.870624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:23.870624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:23.870624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:23.870624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:23.870624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:23.870624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.870624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:23.870624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:23.870624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.870624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:23.870624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:23.870624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.871411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:23.871411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:23.871411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.871411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:28:23.871411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:23.871411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:23.871411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:23.871411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:23.871411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:23.871411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:23.871411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:23.871411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:23.871411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:23.871411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:23.871411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:23.871411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:23.871411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:23.871411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:23.871411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:23.872361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:23.872361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:23.872361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:23.872361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:23.872361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.872361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:23.917381 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:23.918005 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:23.918257 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:23.918464 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:28:23.919930 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:23.919930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.920717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:23.920717 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:23.920717 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:23.920717 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:23.920717 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:23.920717 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:23.920717 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.920717 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:23.921930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.921930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:28:23.921930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:23.921930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:23.921930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:23.921930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:23.921930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:23.921930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:23.921930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:23.921930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:23.921930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:23.921930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:28:23.921930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:23.921930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:28:23.921930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:28:23.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:23.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:28:23.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:23.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:28:23.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:28:23.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:23.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:28:23.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:28:23.923373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.923373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:23.923373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:23.923373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:23.923373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.923373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:28:23.923373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:28:23.923373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:28:23.923373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:23.923373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:23.923373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:23.923373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:23.923837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:23.923837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:23.923837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:23.923837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:23.923837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:23.923837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:23.923837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:23.923837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:23.923837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:23.923837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:23.923837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:23.923837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:23.923837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:23.923837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:23.925338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:23.925338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:23.925338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.925338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:23.925338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.925338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:23.925338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:23.925338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:23.925338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:23.925338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:23.925338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.925338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:23.925338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:23.925338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.926250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:23.926250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:23.926250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.926250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:23.926250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:23.926250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.926250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:28:23.926250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:28:23.926250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:28:23.926250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:23.926250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:23.926250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:23.926250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:23.926250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:23.926250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:23.926250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:23.926250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:23.926250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:23.926250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:23.926976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:23.926976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:23.926976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:23.926976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:23.926976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:23.926976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:23.926976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:23.926976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:23.926976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:23.926976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:23.926976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.926976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:23.932078 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:23.932266 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:23.932368 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:23.932497 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:23.934269 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:23.934269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.934269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:23.934269 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:23.934269 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:23.934269 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:23.934269 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:23.934269 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:23.934269 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.934269 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:23.934755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.934755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:28:23.934755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:23.934755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:23.934755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:23.934755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:23.934755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:23.934755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:23.934755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:23.934755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:23.934755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:23.934755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:28:23.934755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:23.934755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:28:23.934755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:28:23.935356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:23.935356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:28:23.935356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:23.935356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:28:23.935356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:28:23.935356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:23.935356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:28:23.935356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.935356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:28:23.936124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.936124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:23.936124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:23.936124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:23.936124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.936124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:28:23.936124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:28:23.936124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:23.936124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:23.936124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:23.936124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:23.936487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:23.936487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:23.936487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:23.936487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:23.936487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:23.936487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:23.936487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:23.936487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:23.936487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:23.936487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:23.936487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:23.936487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:23.936487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:23.936487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:23.936892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:23.936892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:23.936892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.936892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:23.937994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.937994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:23.937994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:23.937994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:23.937994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:23.937994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:23.937994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.937994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:23.937994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:23.937994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.937994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:23.937994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:23.937994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.937994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:23.937994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:23.937994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.937994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:28:23.938386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:28:23.938386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:23.938386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:23.938386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:23.938386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:23.938386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:23.938386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:23.938386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:23.938386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:23.938386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:23.938386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:23.938386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:23.938386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:23.938386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:23.938386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:23.938386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:23.938386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:23.938386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:23.938386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:23.938767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:23.938767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:23.938767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.938767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:23.941121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:23.941470 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:23.942568 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:23.942568 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:23.942568 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:23.942568 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:23.942568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.943129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:23.943129 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:23.943129 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:23.943129 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:23.943129 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:23.943129 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:23.943129 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.943129 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:23.944654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.944654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:23.944654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:23.944654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:23.944654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.944654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:28:23.944654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:23.944654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:23.944654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:23.944654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:23.944654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:23.944654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:23.944958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:23.944958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:23.944958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:23.944958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:23.944958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:23.944958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:23.944958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:23.944958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:23.944958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:23.944958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:23.944958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:23.944958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:23.944958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:23.945194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.945194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:23.945194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.945194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:23.945194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:23.945194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:23.945194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:23.945194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:23.945194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.945194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:23.945194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:23.945194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.945194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:23.945194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:23.945194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:23.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:23.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:23.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:28:23.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:23.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:23.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:23.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:23.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:23.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:23.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:23.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:23.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:23.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:23.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:23.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:23.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:23.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:23.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:23.945928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:23.945928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:23.945928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:23.945928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:23.945928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:23.945928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:28.629965 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:28.630503 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:28.630634 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:28.631445 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:28.667930 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:28.668094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.668841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:28.668841 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:28.668841 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:28.668841 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:28.668841 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:28.668841 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:28.668841 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.668841 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:28.669476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.669476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:28.669476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:28.669476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:28.669476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.669476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:28:28.669476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:28.669476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:28.669476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:28.669476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:28.669476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:28.669476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:28.670040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:28.670040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:28.670040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:28.670040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:28.670040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:28.670040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:28.670040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:28.670040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:28.670040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:28.670040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:28.670040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:28.670040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:28.670040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:28.670657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.670657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:28.670657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.670657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:28.670657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:28.670657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:28.670657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:28.670657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:28.670657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.670657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:28.670657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:28.670657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.670657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:28.670657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:28.670657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:28.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:28.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:28:28.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:28.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:28.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:28.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:28.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:28.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:28.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:28.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:28.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:28.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:28.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:28.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:28.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:28.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:28.671459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:28.672346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:28.672346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:28.672346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:28.672346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:28.672346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.672346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:28.864089 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:28.864625 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:28.864896 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:28.865795 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:28.866123 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:28.866359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.867274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:28.867274 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:28.867274 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:28.867274 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:28.867274 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:28.867274 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:28.867274 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.867274 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:28.868800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.868800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:28.868800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:28.868800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:28.868800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.868800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:28:28.868800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:28.868800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:28.868800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:28.868800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:28.868800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:28.868800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:28.869378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:28.869378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:28.869378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:28.869378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:28.869378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:28.869378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:28.869378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:28.869378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:28.869378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:28.869378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:28.869378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:28.869378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:28.869378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:28.869986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.869986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:28.870394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.870394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:28.870394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:28.870394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:28.870394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:28.870394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:28.870394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.870394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:28.870394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:28.870394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.870394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:28.870394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:28.870394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.870394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:28.870394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:28.870394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.871109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:28:28.871109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:28.871109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:28.871109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:28.871109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:28.871109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:28.871109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:28.871109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:28.871109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:28.871109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:28.871109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:28.871109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:28.871109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:28.871109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:28.871109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:28.871109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:28.871109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:28.871109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:28.871109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:28.871655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:28.871655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.871655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:28.918222 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:28.918586 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:28.918586 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:28.918947 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:28:28.919362 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:28.919636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.920457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:28.920457 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:28.920457 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:28.920457 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:28.920457 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:28.920457 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:28.920457 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.920457 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:28.921941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.921941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:28:28.921941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:28.921941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:28.921941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:28.921941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:28.921941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:28.921941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:28.921941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:28.921941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:28.921941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:28.921941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:28:28.921941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:28.921941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:28:28.921941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:28:28.922742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:28.922742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:28:28.922742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:28.922742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:28:28.922742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:28:28.922742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:28.922742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:28:28.922742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.922742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:28:28.923275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.923275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:28.923275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:28.923275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:28.923275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.923275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:28:28.923275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:28:28.923275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:28:28.923275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:28.923275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:28.923275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:28.923275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:28.923914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:28.923914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:28.923914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:28.923914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:28.923914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:28.923914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:28.923914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:28.923914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:28.923914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:28.923914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:28.923914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:28.923914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:28.923914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:28.923914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:28.924623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:28.924623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:28.924623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.924623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:28.924623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.924623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:28.924623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:28.924623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:28.924623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:28.924623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:28.924623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.924623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:28.924623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:28.924623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.925400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:28.925400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:28.925400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.925400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:28.925400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:28.925400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.925400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:28:28.925400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:28:28.925400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:28:28.925400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:28.925400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:28.925400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:28.925400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:28.925400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:28.925400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:28.925400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:28.925400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:28.925400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:28.925400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:28.926239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:28.926239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:28.926239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:28.926239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:28.926239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:28.926239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:28.926239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:28.926239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:28.926239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:28.926239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:28.926239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.926239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:28.932378 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:28.932632 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:28.932765 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:28.932888 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:28.933186 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:28.933427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.933856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:28.933856 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:28.933856 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:28.933856 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:28.933856 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:28.933856 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:28.933856 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.933856 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:28.934539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.934539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:28:28.934539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:28.934539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:28.934539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:28.934539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:28.934539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:28.934539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:28.934539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:28.934539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:28.934539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:28.934539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:28:28.934539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:28.934539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:28:28.934539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:28:28.934965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:28.934965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:28:28.934965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:28.934965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:28:28.934965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:28:28.934965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:28.934965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:28:28.934965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.934965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:28:28.935384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.935384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:28.935384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:28.935384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:28.935384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.935384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:28:28.935384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:28:28.935384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:28.935384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:28.935384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:28.935384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:28.935779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:28.935779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:28.935779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:28.935779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:28.935779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:28.935779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:28.935779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:28.935779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:28.935779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:28.935779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:28.935779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:28.935779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:28.935779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:28.935779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:28.936154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:28.936154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:28.936154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.936154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:28.936768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.936768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:28.936768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:28.936768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:28.936768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:28.936768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:28.936768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.936768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:28.936768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:28.936768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.936768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:28.936768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:28.936768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.936768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:28.936768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:28.936768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.936768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:28:28.937244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:28:28.937244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:28.937244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:28.937244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:28.937244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:28.937244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:28.937244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:28.937244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:28.937244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:28.937244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:28.937244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:28.937244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:28.937244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:28.937244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:28.937244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:28.937244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:28.937244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:28.937244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:28.937244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:28.937774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:28.937774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:28.937774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.937774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:28.941642 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:28.942101 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:28.942334 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:28.942613 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:28.943031 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:28.943371 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:28.943590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.944358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:28.944358 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:28.944358 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:28.944358 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:28.944358 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:28.944358 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:28.944358 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.944358 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:28.945696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.945696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:28.945696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:28.945696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:28.945696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.945696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:28:28.945696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:28.945696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:28.945696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:28.945696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:28.945696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:28.945696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:28.946232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:28.946232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:28.946232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:28.946232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:28.946232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:28.946232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:28.946232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:28.946232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:28.946232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:28.946232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:28.946232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:28.946232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:28.946232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:28.946601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.946601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:28.947475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.947475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:28.947475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:28.947475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:28.947475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:28.947475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:28.947475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.947475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:28.947475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:28.947475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.947475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:28.947475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:28.947475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.947475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:28.947475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:28.947475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:28.948005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:28:28.948005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:28.948005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:28.948005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:28.948005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:28.948005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:28.948005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:28.948005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:28.948005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:28.948005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:28.948005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:28.948005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:28.948005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:28.948005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:28.948005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:28.948005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:28.948005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:28.948005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:28.948005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:28.948491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:28.948491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:28.948491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:33.630628 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:33.631316 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:33.631427 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:33.632753 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:33.670116 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:33.670477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.671268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:33.671268 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:33.671268 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:33.671268 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:33.671268 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:33.671268 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:33.671268 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.671268 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:33.672005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.672005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:33.672005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:33.672005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:33.672005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.672005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:28:33.672005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:33.672005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:33.672005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:33.672005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:33.672005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:33.672005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:33.672554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:33.672554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:33.672554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:33.672554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:33.672554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:33.672554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:33.672554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:33.672554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:33.672554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:33.672554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:33.672554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:33.672554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:33.672554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:33.672955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.672955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:33.672955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.672955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:33.672955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:33.672955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:33.672955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:33.672955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:33.672955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.672955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:33.672955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:33.672955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.672955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:33.672955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:33.672955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.673436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:33.673436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:33.673436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.673436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:28:33.673436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:33.673436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:33.673436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:33.673436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:33.673436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:33.673436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:33.673436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:33.673436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:33.673436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:33.673436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:33.673436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:33.673436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:33.673436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:33.673436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:33.673436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:33.674038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:33.674038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:33.674038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:33.674038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:33.674038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.674038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:33.864640 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:33.865210 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:33.865455 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:33.866414 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:33.866623 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:33.866802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.867465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:33.867465 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:33.867465 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:33.867465 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:33.867465 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:33.867465 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:33.867465 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.867465 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:33.868956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.868956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:33.868956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:33.868956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:33.868956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.868956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:28:33.868956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:33.868956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:33.868956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:33.868956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:33.868956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:33.868956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:33.869653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:33.869653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:33.869653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:33.869653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:33.869653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:33.869653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:33.869653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:33.869653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:33.869653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:33.869653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:33.869653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:33.869653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:33.869653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:33.870172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.870172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:33.870542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.870542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:33.870542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:33.870542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:33.870542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:33.870542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:33.870542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.870542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:33.870542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:33.870542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.870542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:33.870542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:33.870542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.870542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:33.870542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:33.870542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.871115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:28:33.871115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:33.871115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:33.871115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:33.871115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:33.871115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:33.871115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:33.871115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:33.871115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:33.871115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:33.871115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:33.871115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:33.871115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:33.871115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:33.871115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:33.871115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:33.871115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:33.871115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:33.871115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:33.871595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:33.871595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.871595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:33.918943 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:33.919472 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:33.919649 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:33.919873 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:28:33.920465 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:33.920861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.921768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:33.921768 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:33.921768 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:33.921768 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:33.921768 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:33.921768 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:33.921768 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.921768 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:33.922993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.922993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:28:33.922993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:33.922993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:33.922993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:33.922993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:33.922993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:33.922993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:33.922993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:33.922993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:33.922993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:33.922993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:28:33.922993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:33.922993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:28:33.922993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:28:33.923516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:33.923516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:28:33.923516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:33.923516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:28:33.923516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:28:33.923516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:33.923516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:28:33.923516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.923516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:28:33.924656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.924656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:33.924656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:33.924656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:33.924656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.924656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:28:33.924656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:28:33.924656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:28:33.924656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:33.924656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:33.924656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:33.924656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:33.925324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:33.925324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:33.925324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:33.925324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:33.925324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:33.925324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:33.925324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:33.925324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:33.925324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:33.925324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:33.925324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:33.925324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:33.925324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:33.925324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:33.925980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:33.925980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:33.925980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.925980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:33.926357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.926357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:33.926357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:33.926357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:33.926357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:33.926357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:33.926357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.926357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:33.926357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:33.926357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.926357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:33.926357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:33.926357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.926357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:33.926357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:33.926357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.927080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:28:33.927080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:28:33.927080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:28:33.927080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:33.927080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:33.927080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:33.927080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:33.927080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:33.927080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:33.927080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:33.927080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:33.927080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:33.927080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:33.927080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:33.927080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:33.927080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:33.927080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:33.927080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:33.927903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:33.927903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:33.927903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:33.927903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:33.927903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:33.927903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.927903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:33.932752 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:33.933021 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:33.933129 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:33.933296 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:33.933690 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:33.934006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.934519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:33.934519 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:33.934519 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:33.934519 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:33.934519 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:33.934519 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:33.934519 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.934519 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:33.935301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.935301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:28:33.935301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:33.935301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:33.935301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:33.935301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:33.935301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:33.935301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:33.935301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:33.935301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:33.935301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:33.935301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:28:33.935301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:33.935301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:28:33.935301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:28:33.936077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:33.936077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:28:33.936077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:33.936077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:28:33.936077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:28:33.936077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:33.936077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:28:33.936077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.936077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:28:33.936531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.936531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:33.936531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:33.936531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:33.936531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.936531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:28:33.936531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:28:33.936531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:33.936531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:33.936531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:33.936531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:33.937121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:33.937121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:33.937121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:33.937121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:33.937121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:33.937121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:33.937121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:33.937121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:33.937121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:33.937121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:33.937121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:33.937121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:33.937121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:33.937121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:33.937857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:33.937857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:33.937857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.937857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:33.938439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.938439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:33.938439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:33.938439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:33.938439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:33.938439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:33.938439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.938439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:33.938439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:33.938439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.938439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:33.938439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:33.938439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.938439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:33.938439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:33.938439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.938439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:28:33.939315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:28:33.939315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:33.939315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:33.939315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:33.939315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:33.939315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:33.939315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:33.939315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:33.939315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:33.939315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:33.939315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:33.939315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:33.939315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:33.939315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:33.939315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:33.939315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:33.939315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:33.939315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:33.939315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:33.940232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:33.940232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:33.940232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.940232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:33.942054 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:33.942362 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:33.942532 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:33.942794 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:33.943101 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:33.943359 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:33.943542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.944117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:33.944117 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:33.944117 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:33.944117 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:33.944117 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:33.944117 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:33.944117 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.944117 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:33.944961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.944961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:33.944961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:33.944961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:33.944961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.944961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:28:33.944961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:33.944961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:33.944961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:33.944961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:33.944961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:33.944961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:33.945347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:33.945347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:33.945347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:33.945347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:33.945347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:33.945347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:33.945347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:33.945347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:33.945347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:33.945347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:33.945347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:33.945347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:33.945347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:33.945696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.945696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:33.946014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.946014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:33.946014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:33.946014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:33.946014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:33.946014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:33.946014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.946014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:33.946014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:33.946014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.946014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:33.946014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:33.946014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.946014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:33.946014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:33.946014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:33.946453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:28:33.946453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:33.946453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:33.946453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:33.946453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:33.946453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:33.946453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:33.946453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:33.946453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:33.946453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:33.946453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:33.946453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:33.946453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:33.946453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:33.946453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:33.946453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:33.946453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:33.946453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:33.946453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:33.946976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:33.946976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:33.946976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:38.631363 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:38.632296 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:38.632296 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:38.633010 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:38.673846 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:38.674076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.674658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:38.674658 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:38.674658 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:38.674658 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:38.674658 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:38.674658 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:38.674658 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.674658 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:38.675768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.675768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:38.675768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:38.675768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:38.675768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.675768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:28:38.675768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:38.675768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:38.675768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:38.675768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:38.675768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:38.675768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:38.676163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:38.676163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:38.676163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:38.676163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:38.676163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:38.676163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:38.676163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:38.676163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:38.676163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:38.676163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:38.676163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:38.676163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:38.676163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:38.676973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.676973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:38.676973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.676973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:38.676973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:38.676973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:38.676973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:38.676973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:38.676973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.676973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:38.676973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:38.676973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.676973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:38.676973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:38.676973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.677524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:38.677524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:38.677524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.677524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:28:38.677524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:38.677524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:38.677524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:38.677524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:38.677524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:38.677524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:38.677524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:38.677524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:38.677524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:38.677524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:38.677524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:38.677524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:38.677524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:38.677524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:38.677524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:38.678123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:38.678123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:38.678123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:38.678123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:38.678123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.678123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:38.865294 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:38.865987 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:38.866293 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:38.867436 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:38.867695 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:38.867960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.868833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:38.868833 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:38.868833 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:38.868833 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:38.868833 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:38.868833 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:38.868833 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.868833 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:38.870739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.870739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:38.870739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:38.870739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:38.870739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.870739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:28:38.870739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:38.870739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:38.870739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:38.870739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:38.870739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:38.870739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:38.872905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:38.872905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:38.872905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:38.872905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:38.872905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:38.872905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:38.872905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:38.872905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:38.872905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:38.872905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:38.872905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:38.872905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:38.872905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:38.873801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.873801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:38.873801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.873801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:38.873801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:38.873801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:38.873801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:38.873801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:38.873801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.873801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:38.873801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:38.873801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.873801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:38.873801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:38.873801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.875287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:38.875287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:38.875287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.875287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:28:38.875287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:38.875287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:38.875287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:38.875287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:38.875287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:38.875287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:38.875287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:38.875287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:38.875287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:38.875287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:38.875287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:38.875287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:38.875287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:38.875287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:38.875287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:38.877345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:38.877345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:38.877345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:38.877345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:38.877345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.877345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:38.919857 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:38.920142 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:38.920374 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:38.920605 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:28:38.921475 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:38.921924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.923159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:38.923159 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:38.923159 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:38.923159 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:38.923159 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:38.923159 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:38.923159 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.923159 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:38.923910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.923910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:28:38.923910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:38.923910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:38.923910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:38.923910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:38.923910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:38.923910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:38.923910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:38.923910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:38.923910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:38.923910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:28:38.923910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:38.923910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:28:38.923910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:28:38.924649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:38.924649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:28:38.924649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:38.924649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:28:38.924649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:28:38.924649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:38.924649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:28:38.924649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.924649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:28:38.925171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.925171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:38.925171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:38.925171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:38.925171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.925171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:28:38.925171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:28:38.925171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:28:38.925171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:38.925171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:38.925171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:38.925171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:38.926082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:38.926082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:38.926082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:38.926082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:38.926082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:38.926082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:38.926082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:38.926082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:38.926082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:38.926082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:38.926082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:38.926082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:38.926082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:38.926082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:38.926758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:38.926758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:38.926758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.926758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:38.926758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.926758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:38.926758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:38.926758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:38.926758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:38.926758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:38.926758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.926758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:38.926758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:38.926758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.927542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:38.927542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:38.927542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.927542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:38.927542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:38.927542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.927542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:28:38.927542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:28:38.927542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:28:38.927542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:38.927542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:38.927542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:38.927542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:38.927542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:38.927542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:38.927542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:38.927542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:38.927542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:38.927542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:38.928246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:38.928246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:38.928246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:38.928246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:38.928246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:38.928246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:38.928246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:38.928246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:38.928246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:38.928246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:38.928246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.928246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:38.933237 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:38.933579 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:38.933728 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:38.933901 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:38.934277 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:38.934533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.935586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:38.935586 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:38.935586 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:38.935586 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:38.935586 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:38.935586 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:38.935586 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.935586 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:38.936355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.936355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:28:38.936355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:38.936355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:38.936355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:38.936355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:38.936355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:38.936355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:38.936355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:38.936355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:38.936355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:38.936355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:28:38.936355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:38.936355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:28:38.936355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:28:38.936857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:38.936857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:28:38.936857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:38.936857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:28:38.936857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:28:38.936857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:38.936857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:28:38.936857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.936857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:28:38.937461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.937461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:38.937461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:38.937461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:38.937461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.937461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:28:38.937461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:28:38.937461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:38.937461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:38.937461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:38.937461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:38.937863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:38.937863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:38.937863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:38.937863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:38.937863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:38.937863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:38.937863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:38.937863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:38.937863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:38.937863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:38.937863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:38.937863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:38.937863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:38.937863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:38.938355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:38.938355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:38.938355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.938355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:38.938908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.938908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:38.938908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:38.938908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:38.938908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:38.938908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:38.938908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.938908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:38.938908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:38.938908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.938908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:38.938908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:38.938908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.938908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:38.938908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:38.938908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.938908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:28:38.939470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:28:38.939470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:38.939470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:38.939470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:38.939470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:38.939470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:38.939470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:38.939470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:38.939470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:38.939470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:38.939470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:38.939470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:38.939470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:38.939470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:38.939470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:38.939470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:38.939470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:38.939470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:38.939470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:38.940216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:38.940216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:38.940216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.940216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:38.942434 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:38.942823 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:38.942956 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:38.943108 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:38.943601 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:38.944126 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:38.944431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.945133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:38.945133 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:38.945133 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:38.945133 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:38.945133 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:38.945133 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:38.945133 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.945133 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:38.946166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.946166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:38.946166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:38.946166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:38.946166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.946166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:28:38.946166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:38.946166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:38.946166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:38.946166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:38.946166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:38.946166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:38.946541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:38.946541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:38.946541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:38.946541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:38.946541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:38.946541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:38.946541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:38.946541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:38.946541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:38.946541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:38.946541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:38.946541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:38.946541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:38.946976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.946976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:38.947413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.947413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:38.947413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:38.947413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:38.947413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:38.947413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:38.947413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.947413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:38.947413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:38.947413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.947413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:38.947413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:38.947413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.947413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:38.947413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:38.947413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:38.947902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:28:38.947902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:38.947902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:38.947902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:38.947902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:38.947902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:38.947902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:38.947902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:38.947902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:38.947902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:38.947902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:38.947902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:38.947902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:38.947902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:38.947902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:38.947902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:38.947902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:38.947902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:38.947902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:38.948426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:38.948426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:38.948426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:43.631987 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:43.632944 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:43.632944 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:43.634251 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:43.665087 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:43.665403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.666321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:43.666321 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:43.666321 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:43.666321 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:43.666321 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:43.666321 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:43.666321 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.666321 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:43.667096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.667096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:43.667096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:43.667096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:43.667096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.667096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:28:43.667096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:43.667096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:43.667096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:43.667096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:43.667096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:43.667096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:43.667735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:43.667735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:43.667735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:43.667735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:43.667735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:43.667735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:43.667735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:43.667735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:43.667735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:43.667735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:43.667735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:43.667735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:43.667735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:43.668361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.668361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:43.668361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.668361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:43.668361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:43.668361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:43.668361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:43.668361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:43.668361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.668361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:43.668361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:43.668361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.668361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:43.668361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:43.668361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.669162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:43.669162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:43.669162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.669162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:28:43.669162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:43.669162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:43.669162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:43.669162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:43.669162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:43.669162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:43.669162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:43.669162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:43.669162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:43.669162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:43.669162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:43.669162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:43.669162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:43.669162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:43.669162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:43.670128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:43.670128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:43.670128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:43.670128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:43.670128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.670128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:43.866770 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:43.867343 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:43.867549 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:43.868773 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:43.869125 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:43.869370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.870321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:43.870321 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:43.870321 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:43.870321 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:43.870321 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:43.870321 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:43.870321 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.870321 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:43.872032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.872032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:43.872032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:43.872032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:43.872032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.872032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:28:43.872032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:43.872032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:43.872032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:43.872032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:43.872032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:43.872032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:43.872727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:43.872727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:43.872727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:43.872727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:43.872727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:43.872727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:43.872727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:43.872727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:43.872727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:43.872727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:43.872727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:43.872727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:43.872727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:43.873367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.873367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:43.874084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.874084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:43.874084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:43.874084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:43.874084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:43.874084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:43.874084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.874084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:43.874084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:43.874084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.874084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:43.874084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:43.874084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.874084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:43.874084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:43.874084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.874942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:28:43.874942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:43.874942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:43.874942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:43.874942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:43.874942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:43.874942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:43.874942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:43.874942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:43.874942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:43.874942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:43.874942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:43.874942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:43.874942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:43.874942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:43.874942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:43.874942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:43.874942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:43.874942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:43.875992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:43.875992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.875992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:43.920082 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:43.920690 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:43.920931 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:43.921126 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:28:43.921999 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:43.922392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.923408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:43.923408 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:43.923408 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:43.923408 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:43.923408 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:43.923408 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:43.923408 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.923408 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:43.924822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.924822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:28:43.924822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:43.924822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:43.924822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:43.924822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:43.924822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:43.924822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:43.924822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:43.924822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:43.924822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:43.924822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:28:43.924822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:43.924822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:28:43.924822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:28:43.926822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:43.926822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:28:43.926822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:43.926822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:28:43.926822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:28:43.926822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:43.926822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:28:43.926822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.926822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:28:43.926822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.926822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:43.926822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:43.926822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:43.926822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.927515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:28:43.927515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:28:43.927515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:28:43.927515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:43.927515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:43.927515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:43.927515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:43.927515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:43.927515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:43.927515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:43.927515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:43.927515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:43.927515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:43.928160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:43.928160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:43.928160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:43.928160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:43.928160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:43.928160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:43.928160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:43.928160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:43.928160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:43.928160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:43.928160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.928160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:43.928992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.928992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:43.928992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:43.928992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:43.928992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:43.928992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:43.928992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.928992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:43.928992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:43.928992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.928992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:43.928992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:43.928992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.928992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:43.928992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:43.928992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.929463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:28:43.929463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:28:43.929463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:28:43.929463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:43.929463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:43.929463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:43.929463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:43.929463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:43.929463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:43.929463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:43.929463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:43.929463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:43.929463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:43.929463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:43.929463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:43.929463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:43.929463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:43.929463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:43.930028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:43.930028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:43.930028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:43.930028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:43.930028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:43.930028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.930028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:43.933647 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:43.933922 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:43.934150 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:43.934150 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:43.934818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:43.935017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.935516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:43.935516 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:43.935516 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:43.935516 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:43.935516 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:43.935516 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:43.935516 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.935516 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:43.936259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.936259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:28:43.936259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:43.936259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:43.936259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:43.936259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:43.936259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:43.936259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:43.936259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:43.936259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:43.936259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:43.936259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:28:43.936259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:43.936259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:28:43.936259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:28:43.936895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:43.936895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:28:43.936895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:43.936895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:28:43.936895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:28:43.936895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:43.936895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:28:43.936895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.936895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:28:43.937622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.937622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:43.937622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:43.937622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:43.937622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.937622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:28:43.937622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:28:43.937622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:43.937622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:43.937622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:43.937622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:43.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:43.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:43.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:43.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:43.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:43.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:43.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:43.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:43.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:43.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:43.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:43.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:43.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:43.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:43.938370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:43.938370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:43.938370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.938370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:43.939465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.939465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:43.939465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:43.939465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:43.939465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:43.939465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:43.939465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.939465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:43.939465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:43.939465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.939465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:43.939465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:43.939465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.939465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:43.939465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:43.939465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.939465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:28:43.939962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:28:43.939962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:43.939962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:43.939962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:43.939962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:43.939962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:43.939962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:43.939962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:43.939962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:43.939962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:43.939962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:43.939962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:43.939962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:43.939962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:43.939962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:43.939962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:43.939962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:43.939962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:43.939962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:43.940499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:43.940499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:43.940499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.940499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:43.942801 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:43.943147 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:43.943326 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:43.943564 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:43.943939 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:43.944195 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:43.944377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.944940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:43.944940 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:43.944940 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:43.944940 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:43.944940 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:43.944940 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:43.944940 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.944940 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:43.945771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.945771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:43.945771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:43.945771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:43.945771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.945771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:28:43.945771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:43.945771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:43.945771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:43.945771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:43.945771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:43.945771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:43.946125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:43.946125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:43.946125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:43.946125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:43.946125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:43.946125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:43.946125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:43.946125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:43.946125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:43.946125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:43.946125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:43.946125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:43.946125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:43.946474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.946474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:43.946789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.946789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:43.946789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:43.946789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:43.946789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:43.946789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:43.946789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.946789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:43.946789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:43.946789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.946789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:43.946789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:43.946789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.946789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:43.946789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:43.946789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:43.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:28:43.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:43.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:43.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:43.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:43.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:43.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:43.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:43.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:43.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:43.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:43.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:43.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:43.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:43.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:43.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:43.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:43.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:43.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:43.947733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:43.947733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:43.947733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:48.632640 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:48.633464 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:48.633464 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:48.634452 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:48.668094 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:48.668358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.669128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:48.669128 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:48.669128 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:48.669128 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:48.669128 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:48.669128 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:48.669128 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.669128 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:48.669774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.669774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:48.669774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:48.669774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:48.669774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.669774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:28:48.669774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:48.669774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:48.669774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:48.669774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:48.669774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:48.669774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:48.670381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:48.670381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:48.670381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:48.670381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:48.670381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:48.670381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:48.670381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:48.670381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:48.670381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:48.670381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:48.670381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:48.670381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:48.670381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:48.671000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.671000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:48.671000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.671000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:48.671000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:48.671000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:48.671000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:48.671000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:48.671000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.671000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:48.671000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:48.671000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.671000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:48.671000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:48.671000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.671699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:48.671699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:48.671699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.671699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:28:48.671699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:48.671699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:48.671699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:48.671699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:48.671699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:48.671699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:48.671699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:48.671699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:48.671699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:48.671699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:48.671699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:48.671699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:48.671699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:48.671699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:48.671699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:48.672511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:48.672511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:48.672511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:48.672511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:48.672511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.672511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:48.867243 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:48.867722 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:48.867836 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:48.868705 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:48.869032 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:48.869255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.870055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:48.870055 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:48.870055 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:48.870055 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:48.870055 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:48.870055 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:48.870055 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.870055 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:48.871177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.871177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:48.871177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:48.871177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:48.871177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.871177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:28:48.871177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:48.871177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:48.871177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:48.871177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:48.871177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:48.871177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:48.871971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:48.871971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:48.871971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:48.871971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:48.871971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:48.871971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:48.871971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:48.871971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:48.871971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:48.871971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:48.871971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:48.871971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:48.871971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:48.872767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.872767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:48.872767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.872767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:48.872767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:48.872767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:48.872767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:48.872767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:48.872767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.872767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:48.872767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:48.872767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.872767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:48.872767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:48.872767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.873454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:48.873454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:48.873454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.873454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:28:48.873454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:48.873454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:48.873454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:48.873454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:48.873454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:48.873454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:48.873454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:48.873454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:48.873454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:48.873454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:48.873454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:48.873454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:48.873454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:48.873454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:48.873454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:48.874140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:48.874140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:48.874140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:48.874140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:48.874140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.874140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:48.920764 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:48.921256 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:48.921466 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:48.921696 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:28:48.922389 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:48.922701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.923654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:48.923654 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:48.923654 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:48.923654 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:48.923654 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:48.923654 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:48.923654 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.923654 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:48.925802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.925802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:28:48.925802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:48.925802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:48.925802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:48.925802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:48.925802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:48.925802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:48.925802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:48.925802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:48.925802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:48.925802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:28:48.925802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:48.925802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:28:48.925802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:28:48.926233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:48.926233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:28:48.926233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:48.926233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:28:48.926233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:28:48.926233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:48.926233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:28:48.926233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.926233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:28:48.927285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.927285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:48.927285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:48.927285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:48.927285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.927285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:28:48.927285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:28:48.927285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:28:48.927285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:48.927285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:48.927285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:48.927285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:48.927643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:48.927643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:48.927643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:48.927643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:48.927643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:48.927643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:48.927643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:48.927643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:48.927643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:48.927643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:48.927643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:48.927643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:48.927643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:48.927643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:48.928067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:48.928067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:48.928067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.928067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:48.929059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.929059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:48.929059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:48.929059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:48.929059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:48.929059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:48.929059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.929059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:48.929059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:48.929059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.929059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:48.929059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:48.929059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.929059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:48.929059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:48.929059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.929456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:28:48.929456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:28:48.929456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:28:48.929456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:48.929456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:48.929456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:48.929456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:48.929456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:48.929456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:48.929456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:48.929456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:48.929456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:48.929456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:48.929456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:48.929456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:48.929456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:48.929456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:48.929456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:48.929960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:48.929960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:48.929960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:48.929960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:48.929960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:48.929960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.929960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:48.934140 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:48.934433 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:48.934547 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:48.934664 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:48.935227 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:48.935636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.936195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:48.936195 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:48.936195 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:48.936195 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:48.936195 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:48.936195 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:48.936195 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.936195 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:48.936972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.936972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:28:48.936972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:48.936972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:48.936972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:48.936972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:48.936972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:48.936972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:48.936972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:48.936972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:48.936972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:48.936972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:28:48.936972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:48.936972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:28:48.936972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:28:48.937366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:48.937366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:28:48.937366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:48.937366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:28:48.937366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:28:48.937366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:48.937366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:28:48.937366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.937366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:28:48.937834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.937834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:48.937834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:48.937834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:48.937834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.937834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:28:48.937834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:28:48.937834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:48.937834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:48.937834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:48.937834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:48.938166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:48.938166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:48.938166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:48.938166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:48.938166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:48.938166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:48.938166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:48.938166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:48.938166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:48.938166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:48.938166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:48.938166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:48.938166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:48.938166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:48.938550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:48.938550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:48.938550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.938550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:48.938957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.938957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:48.938957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:48.938957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:48.938957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:48.938957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:48.938957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.938957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:48.938957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:48.938957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.938957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:48.938957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:48.938957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.938957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:48.938957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:48.938957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.938957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:28:48.939430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:28:48.939430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:48.939430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:48.939430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:48.939430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:48.939430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:48.939430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:48.939430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:48.939430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:48.939430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:48.939430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:48.939430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:48.939430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:48.939430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:48.939430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:48.939430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:48.939430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:48.939430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:48.939430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:48.939989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:48.939989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:48.939989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.939989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:48.943209 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:48.943506 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:48.943656 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:48.943885 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:48.944315 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:48.944657 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:48.944921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.945625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:48.945625 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:48.945625 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:48.945625 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:48.945625 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:48.945625 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:48.945625 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.945625 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:48.946493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.946493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:48.946493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:48.946493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:48.946493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.946493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:28:48.946493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:48.946493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:48.946493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:48.946493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:48.946493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:48.946493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:48.946904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:48.946904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:48.946904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:48.946904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:48.946904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:48.946904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:48.946904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:48.946904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:48.946904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:48.946904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:48.946904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:48.946904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:48.946904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:48.947264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.947264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:48.947449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.947449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:48.947449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:48.947449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:48.947449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:48.947449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:48.947449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.947449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:48.947449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:48.947449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.947449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:48.947449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:48.947449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.947449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:48.947449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:48.947449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:48.947941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:28:48.947941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:48.947941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:48.947941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:48.947941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:48.947941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:48.947941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:48.947941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:48.947941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:48.947941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:48.947941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:48.947941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:48.947941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:48.947941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:48.947941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:48.947941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:48.947941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:48.947941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:48.947941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:48.948522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:48.948522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:48.948522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:53.633183 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:53.633834 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:53.633834 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:53.634591 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:53.667748 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:53.668266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.669148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:53.669148 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:53.669148 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:53.669148 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:53.669148 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:53.669148 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:53.669148 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.669148 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:53.670604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.670604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:53.670604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:53.670604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:53.670604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.670604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:28:53.670604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:53.670604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:53.670604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:53.670604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:53.670604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:53.670604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:53.671277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:53.671277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:53.671277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:53.671277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:53.671277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:53.671277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:53.671277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:53.671277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:53.671277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:53.671277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:53.671277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:53.671277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:53.671277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:53.671912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.671912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:53.671912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.671912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:53.671912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:53.671912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:53.671912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:53.671912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:53.671912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.671912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:53.671912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:53.671912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.671912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:53.671912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:53.671912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.672658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:53.672658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:53.672658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.672658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:28:53.672658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:53.672658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:53.672658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:53.672658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:53.672658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:53.672658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:53.672658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:53.672658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:53.672658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:53.672658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:53.672658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:53.672658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:53.672658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:53.672658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:53.672658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:53.673576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:53.673576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:53.673576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:53.673576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:53.673576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.673576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:53.867791 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:53.868232 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:53.868414 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:53.869337 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:53.869570 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:53.869793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.870475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:53.870475 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:53.870475 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:53.870475 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:53.870475 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:53.870475 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:53.870475 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.870475 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:53.871852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.871852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:53.871852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:53.871852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:53.871852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.871852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:28:53.871852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:53.871852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:53.871852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:53.871852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:53.871852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:53.871852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:53.872263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:53.872263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:53.872263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:53.872263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:53.872263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:53.872263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:53.872263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:53.872263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:53.872263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:53.872263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:53.872263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:53.872263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:53.872263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:53.872647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.872647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:53.873825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.873825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:53.873825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:53.873825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:53.873825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:53.873825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:53.873825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.873825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:53.873825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:53.873825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.873825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:53.873825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:53.873825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.873825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:53.873825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:53.873825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.874722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:28:53.874722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:53.874722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:53.874722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:53.874722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:53.874722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:53.874722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:53.874722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:53.874722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:53.874722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:53.874722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:53.874722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:53.874722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:53.874722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:53.874722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:53.874722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:53.874722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:53.874722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:53.874722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:53.875577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:53.875577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.875577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:53.921304 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:53.922309 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:53.922309 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:53.922309 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:28:53.922664 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:53.923017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.926706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:53.926706 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:53.926706 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:53.926706 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:53.926706 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:53.926706 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:53.926706 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.926706 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:53.926706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.926706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:28:53.926706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:53.926706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:53.926706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:53.926706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:53.926706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:53.926706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:53.926706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:53.926706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:53.927593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:53.927593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:28:53.927593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:53.927593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:28:53.927593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:28:53.927593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:53.927593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:28:53.927593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:53.927593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:28:53.927593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:28:53.927593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:53.927593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:28:53.927593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.927593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:28:53.927593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.928324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:53.928324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:53.928324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:53.928324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.928324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:28:53.928324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:28:53.928324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:28:53.928324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:53.928324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:53.928324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:53.928324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:53.928324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:53.928324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:53.928943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:53.928943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:53.928943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:53.928943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:53.928943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:53.928943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:53.928943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:53.928943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:53.928943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:53.928943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:53.928943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:53.928943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:53.928943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:53.928943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:53.929515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.929515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:53.929515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.929515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:53.929515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:53.929515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:53.929515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:53.929515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:53.929515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.929515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:53.929515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:53.929515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.929515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:53.929515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:53.929515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.930230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:53.930230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:53.930230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.930230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:28:53.930230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:28:53.930230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:28:53.930230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:53.930230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:53.930230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:53.930230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:53.930230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:53.930230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:53.930230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:53.930230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:53.930230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:53.930230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:53.930230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:53.930230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:53.930230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:53.931052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:53.931052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:53.931052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:53.931052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:53.931052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:53.931052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:53.931052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:53.931052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.931052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:53.934510 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:53.934849 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:53.934970 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:53.935115 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:53.935561 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:53.935950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.936585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:53.936585 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:53.936585 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:53.936585 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:53.936585 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:53.936585 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:53.936585 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.936585 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:53.937429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.937429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:28:53.937429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:53.937429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:53.937429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:53.937429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:53.937429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:53.937429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:53.937429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:53.937429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:53.937429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:53.937429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:28:53.937429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:53.937429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:28:53.937429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:28:53.937886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:53.937886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:28:53.937886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:53.937886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:28:53.937886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:28:53.937886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:53.937886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:28:53.937886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.937886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:28:53.938506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.938506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:53.938506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:53.938506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:53.938506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.938506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:28:53.938506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:28:53.938506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:53.938506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:53.938506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:53.938506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:53.938883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:53.938883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:53.938883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:53.938883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:53.938883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:53.938883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:53.938883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:53.938883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:53.938883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:53.938883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:53.938883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:53.938883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:53.938883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:53.938883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:53.939291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:53.939291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:53.939291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.939291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:53.943352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.943352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:53.943352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:53.943352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:53.943352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:53.943352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:53.943352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.943352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:53.943352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:53.943352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.943352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:53.943352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:53.943352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.943352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:53.943352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:53.943352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.943352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:28:53.944151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:28:53.944151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:53.944151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:53.944151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:53.944151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:53.944151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:53.944151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:53.944151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:53.944151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:53.944151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:53.944151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:53.944151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:53.944151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:53.944151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:53.944151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:53.944151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:53.944151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:53.944151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:53.944151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:53.944718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:53.944718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:53.944718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.944718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:53.946084 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:53.946412 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:53.946574 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:53.946843 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:53.947265 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:53.947588 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:53.947828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.948523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:53.948523 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:53.948523 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:53.948523 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:53.948523 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:53.948523 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:53.948523 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.948523 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:53.949503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.949503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:53.949503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:53.949503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:53.949503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.949503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:28:53.949503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:53.949503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:53.949503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:53.949503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:53.949503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:53.949503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:53.949925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:53.949925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:53.949925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:53.949925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:53.949925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:53.949925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:53.949925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:53.949925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:53.949925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:53.949925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:53.949925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:53.949925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:53.949925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:53.950281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.950281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:53.950634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.950634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:53.950634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:53.950634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:53.950634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:53.950634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:53.950634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.950634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:53.950634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:53.950634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.950634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:53.950634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:53.950634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.950634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:53.950634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:53.950634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:53.951171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:28:53.951171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:53.951171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:53.951171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:53.951171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:53.951171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:53.951171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:53.951171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:53.951171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:53.951171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:53.951171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:53.951171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:53.951171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:53.951171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:53.951171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:53.951171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:53.951171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:53.951171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:53.951171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:53.951738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:53.951738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:53.951738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:58.633628 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:58.634366 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:58.634366 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:58.634861 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:58.668175 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:58.668691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.669504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:58.669504 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:58.669504 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:58.669504 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:58.669504 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:58.669504 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:58.669504 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.669504 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:58.670306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.670306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:58.670306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:58.670306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:58.670306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.670306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:28:58.670306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:58.670306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:58.670306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:58.670306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:58.670306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:58.670306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:58.671014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:58.671014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:58.671014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:58.671014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:58.671014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:58.671014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:58.671014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:58.671014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:58.671014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:58.671014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:58.671014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:58.671014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:58.671014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:58.671605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.671605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:58.671605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.671605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:58.671605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:58.671605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:58.671605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:58.671605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:58.671605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.671605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:58.671605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:58.671605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.671605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:58.671605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:58.671605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.672331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:58.672331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:58.672331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.672331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:28:58.672331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:58.672331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:58.672331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:58.672331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:58.672331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:58.672331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:58.672331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:58.672331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:58.672331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:58.672331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:58.672331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:58.672331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:58.672331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:58.672331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:58.672331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:58.673183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:58.673183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:58.673183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:58.673183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:58.673183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.673183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:58.868869 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:58.870337 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:58.870647 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:58.871698 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:58.871922 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:58.872033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.873467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:58.873467 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:58.873467 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:58.873467 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:58.873467 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:58.873467 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:58.873467 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.873467 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:58.875137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.875137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:58.875137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:58.875137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:58.875137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.875137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:28:58.875137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:58.875137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:58.875137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:58.875137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:58.875137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:58.875137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:58.875976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:58.875976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:58.875976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:58.875976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:58.875976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:58.875976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:58.875976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:58.875976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:58.875976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:58.875976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:58.875976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:58.875976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:58.875976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:58.876561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.876561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:58.877337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.877337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:58.877337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:58.877337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:58.877337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:58.877337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:58.877337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.877337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:58.877337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:58.877337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.877337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:58.877337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:58.877337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.877337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:58.877337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:58.877337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.877962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:28:58.877962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:58.877962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:58.877962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:58.877962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:58.877962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:58.877962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:58.877962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:58.877962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:58.877962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:58.877962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:58.877962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:58.877962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:58.877962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:58.877962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:58.877962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:58.877962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:58.877962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:58.877962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:58.878539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:58.878539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.878539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:58.922231 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:58.922667 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:58.922953 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:58.923215 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:28:58.923822 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:58.924111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.924909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:58.924909 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:58.924909 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:58.924909 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:58.924909 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:58.924909 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:58.924909 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.924909 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:58.926296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.926296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:28:58.926296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:58.926296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:58.926296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:58.926296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:58.926296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:58.926296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:58.926296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:58.926296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:58.926296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:58.926296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:28:58.926296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:58.926296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:28:58.926296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:28:58.927427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:58.927427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:28:58.927427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:58.927427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:28:58.927427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:28:58.927427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:58.927427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:28:58.927427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.927427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:28:58.928137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.928137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:58.928137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:58.928137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:58.928137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.928137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:28:58.928137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:28:58.928137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:28:58.928137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:58.928137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:58.928137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:58.928137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:58.929309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:58.929309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:58.929309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:58.929309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:58.929309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:58.929309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:58.929309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:58.929309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:58.929309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:58.929309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:58.929309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:58.929309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:58.929309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:58.929309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:58.930124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:58.930124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:58.930124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.930124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:58.930385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.930385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:58.930385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:58.930385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:58.930385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:58.930385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:58.930385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.930385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:58.930385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:58.930385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.930385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:58.930385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:58.930385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.930385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:58.930385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:58.930385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.930943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:28:58.930943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:28:58.930943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:28:58.930943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:58.930943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:58.930943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:58.930943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:58.930943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:58.930943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:58.930943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:58.930943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:58.930943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:58.930943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:58.930943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:58.930943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:58.930943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:58.930943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:58.930943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:58.931656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:58.931656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:58.931656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:58.931656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:58.931656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:58.931656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.931656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:58.934990 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:58.935244 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:58.935336 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:58.935461 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:58.935955 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:58.936305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.937002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:58.937002 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:58.937002 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:58.937002 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:58.937002 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:58.937002 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:58.937002 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.937002 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:58.937576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.937576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:28:58.937576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:58.937576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:58.937576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:58.937576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:58.937576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:58.937576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:58.937576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:58.937576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:28:58.937576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:58.937576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:28:58.937576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:58.937576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:28:58.937576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:28:58.938025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:58.938025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:28:58.938025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:58.938025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:28:58.938025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:28:58.938025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:58.938025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:28:58.938025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.938025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:28:58.938865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.938865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:58.938865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:58.938865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:58.938865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.938865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:28:58.938865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:28:58.938865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:58.938865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:58.938865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:58.938865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:58.939206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:58.939206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:58.939206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:58.939206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:58.939206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:58.939206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:58.939206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:58.939206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:58.939206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:58.939206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:58.939206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:58.939206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:58.939206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:58.939206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:58.939578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:58.939578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:58.939578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.939578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:58.939578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.939578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:58.939578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:58.939578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:58.939578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:58.939578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:58.939578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.939578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:58.939578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:58.939578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.940150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:58.940150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:58.940150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.940150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:58.940150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:58.940150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.940150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:28:58.940150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:28:58.940150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:28:58.940150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:28:58.940150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:58.940150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:58.940150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:58.940150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:58.940150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:58.940150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:58.940150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:58.940150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:58.940150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:58.940767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:58.940767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:58.940767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:58.940767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:58.940767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:58.940767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:58.940767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:58.940767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:58.940767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:58.940767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.940767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:28:58.946478 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:28:58.946762 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:28:58.946906 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:28:58.947093 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:28:58.947513 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:58.947874 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:28:58.948083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.948794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:28:58.948794 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:58.948794 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:28:58.948794 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:28:58.948794 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:58.948794 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:28:58.948794 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.948794 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:28:58.949939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.949939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:28:58.949939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:58.949939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:28:58.949939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.949939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:28:58.949939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:58.949939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:58.949939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:58.949939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:58.949939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:58.949939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:58.950294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:28:58.950294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:58.950294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:28:58.950294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:58.950294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:28:58.950294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:28:58.950294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:58.950294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:28:58.950294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:58.950294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:28:58.950294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:28:58.950294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:58.950294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:28:58.950635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.950635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:28:58.951686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.951686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:28:58.951686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:28:58.951686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:28:58.951686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:28:58.951686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:28:58.951686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.951686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:28:58.951686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:28:58.951686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.951686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:28:58.951686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:28:58.951686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.951686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:28:58.951686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:28:58.951686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:28:58.952188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:28:58.952188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:28:58.952188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:58.952188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:58.952188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:28:58.952188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:28:58.952188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:28:58.952188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:28:58.952188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:28:58.952188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:28:58.952188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:28:58.952188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:28:58.952188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:28:58.952188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:28:58.952188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:28:58.952188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:28:58.952188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:28:58.952188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:28:58.952188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:28:58.952808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:28:58.952808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:28:58.952808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:03.634513 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:03.635415 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:03.635415 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:03.635992 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:03.641647 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=240 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:29:03.671490 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:03.671941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.672784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:03.672784 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:03.672784 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:03.672784 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:03.672784 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:03.672784 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:03.672784 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.672784 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:03.673979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.673979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:03.673979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:03.673979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:03.673979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.673979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:29:03.673979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:03.673979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:03.673979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:03.673979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:03.673979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:03.673979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:03.674634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:03.674634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:03.674634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:03.674634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:03.674634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:03.674634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:03.674634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:03.674634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:03.674634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:03.674634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:03.674634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:03.674634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:03.674634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:03.675435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.675435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:03.675435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.675435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:03.675435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:03.675435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:03.675435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:03.675435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:03.675435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.675435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:03.675435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:03.675435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.675435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:03.675435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:03.675435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.676962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:03.676962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:03.676962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.676962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:29:03.676962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:03.676962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:03.676962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:03.676962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:03.676962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:03.676962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:03.676962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:03.676962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:03.676962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:03.676962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:03.676962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:03.676962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:03.676962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:03.676962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:03.676962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:03.678240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:03.678240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:03.678240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:03.678240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:03.678240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.678240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:03.870461 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:03.870943 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:03.871128 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:03.872264 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:03.872625 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:03.872909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.873772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:03.873772 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:03.873772 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:03.873772 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:03.873772 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:03.873772 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:03.873772 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.873772 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:03.875264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.875264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:03.875264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:03.875264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:03.875264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.875264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:29:03.875264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:03.875264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:03.875264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:03.875264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:03.875264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:03.875264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:03.875897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:03.875897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:03.875897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:03.875897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:03.875897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:03.875897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:03.875897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:03.875897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:03.875897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:03.875897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:03.875897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:03.875897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:03.875897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:03.876420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.876420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:03.876990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.876990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:03.876990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:03.876990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:03.876990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:03.876990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:03.876990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.876990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:03.876990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:03.876990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.876990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:03.876990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:03.876990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.876990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:03.876990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:03.876990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.877532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:29:03.877532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:03.877532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:03.877532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:03.877532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:03.877532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:03.877532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:03.877532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:03.877532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:03.877532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:03.877532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:03.877532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:03.877532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:03.877532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:03.877532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:03.877532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:03.877532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:03.877532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:03.877532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:03.878039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:03.878039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.878039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:03.923212 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:03.923529 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:03.923731 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:03.923929 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:29:03.924655 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:03.925003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.925838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:03.925838 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:03.925838 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:03.925838 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:03.925838 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:03.925838 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:03.925838 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.925838 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:03.926986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.926986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:29:03.926986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:03.926986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:03.926986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:03.926986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:03.926986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:03.926986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:03.926986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:03.926986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:03.926986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:03.926986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:29:03.926986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:03.926986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:29:03.926986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:29:03.928367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:03.928367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:29:03.928367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:03.928367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:29:03.928367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:29:03.928367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:03.928367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:29:03.928367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.928367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:29:03.928367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.928367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:03.928367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:03.928367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:03.928367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.928998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:29:03.928998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:29:03.928998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:29:03.928998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:03.928998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:03.928998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:03.928998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:03.928998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:03.928998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:03.928998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:03.928998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:03.928998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:03.928998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:03.929517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:03.929517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:03.929517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:03.929517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:03.929517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:03.929517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:03.929517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:03.929517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:03.929517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:03.929517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:03.929517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.929517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:03.930022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.930022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:03.930022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:03.930022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:03.930022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:03.930022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:03.930022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.930022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:03.930022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:03.930022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.930022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:03.930022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:03.930022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.930022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:03.930022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:03.930022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.930630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:29:03.930630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:29:03.930630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:29:03.930630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:03.930630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:03.930630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:03.930630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:03.930630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:03.930630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:03.930630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:03.930630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:03.930630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:03.930630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:03.930630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:03.930630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:03.930630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:03.930630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:03.930630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:03.931393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:03.931393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:03.931393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:03.931393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:03.931393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:03.931393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.931393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:03.935354 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:03.935591 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:03.935722 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:03.935851 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:03.936319 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:03.936668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.937340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:03.937340 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:03.937340 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:03.937340 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:03.937340 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:03.937340 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:03.937340 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.937340 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:03.938116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.938116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:29:03.938116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:03.938116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:03.938116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:03.938116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:03.938116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:03.938116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:03.938116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:03.938116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:03.938116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:03.938116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:29:03.938116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:03.938116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:29:03.938116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:29:03.938590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:03.938590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:29:03.938590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:03.938590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:29:03.938590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:29:03.938590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:03.938590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:29:03.938590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.938590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:29:03.939065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.939065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:03.939065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:03.939065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:03.939065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.939065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:29:03.939065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:29:03.939065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:03.939065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:03.939065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:03.939065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:03.939410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:03.939410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:03.939410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:03.939410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:03.939410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:03.939410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:03.939410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:03.939410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:03.939410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:03.939410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:03.939410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:03.939410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:03.939410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:03.939410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:03.939842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:03.939842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:03.939842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.939842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:03.940158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.940158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:03.940158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:03.940158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:03.940158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:03.940158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:03.940158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.940158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:03.940158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:03.940158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.940158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:03.940158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:03.940158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.940158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:03.940158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:03.940158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.940158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:29:03.940668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:29:03.940668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:03.940668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:03.940668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:03.940668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:03.940668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:03.940668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:03.940668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:03.940668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:03.940668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:03.940668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:03.940668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:03.940668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:03.940668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:03.940668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:03.940668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:03.940668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:03.940668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:03.940668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:03.941243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:03.941243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:03.941243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.941243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:03.946816 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:03.947036 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:03.947145 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:03.947295 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:03.947784 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:03.948164 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:03.948415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.949114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:03.949114 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:03.949114 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:03.949114 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:03.949114 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:03.949114 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:03.949114 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.949114 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:03.950284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.950284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:03.950284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:03.950284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:03.950284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.950284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:29:03.950284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:03.950284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:03.950284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:03.950284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:03.950284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:03.950284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:03.950883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:03.950883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:03.950883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:03.950883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:03.950883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:03.950883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:03.950883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:03.950883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:03.950883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:03.950883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:03.950883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:03.950883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:03.950883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:03.951450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.951450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:03.951789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.951789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:03.951789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:03.951789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:03.951789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:03.951789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:03.951789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.951789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:03.951789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:03.951789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.951789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:03.951789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:03.951789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.951789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:03.951789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:03.951789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:03.952505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:29:03.952505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:03.952505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:03.952505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:03.952505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:03.952505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:03.952505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:03.952505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:03.952505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:03.952505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:03.952505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:03.952505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:03.952505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:03.952505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:03.952505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:03.952505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:03.952505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:03.952505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:03.952505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:03.953349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:03.953349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:03.953349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:08.635169 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:08.636024 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:08.636024 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:08.637360 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:08.673350 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:08.673785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.674735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:08.674735 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:08.674735 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:08.674735 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:08.674735 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:08.674735 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:08.674735 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.674735 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:08.675521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.675521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:08.675521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:08.675521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:08.675521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.675521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:29:08.675521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:08.675521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:08.675521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:08.675521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:08.675521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:08.675521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:08.675908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:08.675908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:08.675908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:08.675908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:08.675908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:08.675908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:08.675908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:08.675908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:08.675908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:08.675908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:08.675908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:08.675908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:08.675908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:08.676286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.676286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:08.676286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.676286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:08.676286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:08.676286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:08.676286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:08.676286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:08.676286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.676286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:08.676286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:08.676286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.676286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:08.676286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:08.676286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.676867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:08.676867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:08.676867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.676867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:29:08.676867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:08.676867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:08.676867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:08.676867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:08.676867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:08.676867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:08.676867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:08.676867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:08.676867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:08.676867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:08.676867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:08.676867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:08.676867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:08.676867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:08.676867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:08.677467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:08.677467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:08.677467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:08.677467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:08.677467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.677467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:08.871164 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:08.871869 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:08.872067 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:08.873574 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:08.874028 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:08.874982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.875404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:08.875404 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:08.875404 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:08.875404 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:08.875404 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:08.875404 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:08.875404 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.875404 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:08.877130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.877130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:08.877130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:08.877130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:08.877130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.877130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:29:08.877130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:08.877130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:08.877130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:08.877130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:08.877130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:08.877130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:08.877601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:08.877601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:08.877601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:08.877601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:08.877601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:08.877601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:08.877601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:08.877601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:08.877601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:08.877601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:08.877601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:08.877601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:08.877601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:08.877954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.877954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:08.878309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.878309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:08.878309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:08.878309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:08.878309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:08.878309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:08.878309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.878309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:08.878309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:08.878309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.878309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:08.878309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:08.878309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.878309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:08.878309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:08.878309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.879880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:29:08.879880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:08.879880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:08.879880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:08.879880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:08.879880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:08.879880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:08.879880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:08.879880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:08.879880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:08.879880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:08.879880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:08.879880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:08.879880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:08.879880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:08.879880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:08.879880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:08.879880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:08.879880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:08.880839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:08.880839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.880839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:08.923879 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:08.924462 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:08.924715 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:08.924940 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:29:08.926077 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:08.926522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.927752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:08.927752 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:08.927752 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:08.927752 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:08.927752 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:08.927752 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:08.927752 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.927752 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:08.929348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.929348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:29:08.929348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:08.929348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:08.929348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:08.929348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:08.929348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:08.929348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:08.929348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:08.929348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:08.929348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:08.929348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:29:08.929348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:08.929348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:29:08.929348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:29:08.929991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:08.929991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:29:08.929991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:08.929991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:29:08.929991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:29:08.929991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:08.929991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:29:08.929991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.929991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:29:08.930974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.930974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:08.930974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:08.930974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:08.930974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.930974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:29:08.930974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:29:08.930974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:29:08.930974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:08.930974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:08.930974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:08.930974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:08.932086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:08.932086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:08.932086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:08.932086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:08.932086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:08.932086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:08.932086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:08.932086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:08.932086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:08.932086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:08.932086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:08.932086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:08.932086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:08.932086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:08.933060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:08.933060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:08.933060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.933060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:08.933060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.933060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:08.933060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:08.933060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:08.933060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:08.933060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:08.933060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.933060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:08.933060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:08.933060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.933571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:08.933571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:08.933571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.933571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:08.933571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:08.933571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.933571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:29:08.933571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:29:08.933571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:29:08.933571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:08.933571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:08.933571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:08.933571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:08.933571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:08.933571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:08.933571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:08.933571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:08.933571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:08.933571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:08.934101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:08.934101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:08.934101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:08.934101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:08.934101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:08.934101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:08.934101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:08.934101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:08.934101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:08.934101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:08.934101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.934101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:08.935813 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:08.936223 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:08.936223 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:08.936374 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:08.937279 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:08.937661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.938741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:08.938741 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:08.938741 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:08.938741 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:08.938741 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:08.938741 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:08.938741 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.938741 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:08.939238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.939238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:29:08.939238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:08.939238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:08.939238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:08.939238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:08.939238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:08.939238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:08.939238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:08.939238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:08.939238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:08.939238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:29:08.939238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:08.939238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:29:08.939238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:29:08.939710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:08.939710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:29:08.939710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:08.939710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:29:08.939710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:29:08.939710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:08.939710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:29:08.939710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.939710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:29:08.940081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.940081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:08.940081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:08.940081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:08.940081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.940081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:29:08.940081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:29:08.940081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:08.940081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:08.940081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:08.940081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:08.940416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:08.940416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:08.940416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:08.940416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:08.940416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:08.940416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:08.940416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:08.940416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:08.940416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:08.940416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:08.940416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:08.940416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:08.940416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:08.940416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:08.940908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:08.940908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:08.940908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.940908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:08.941188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.941188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:08.941188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:08.941188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:08.941188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:08.941188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:08.941188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.941188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:08.941188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:08.941188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.941188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:08.941188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:08.941188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.941188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:08.941188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:08.941188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.941188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:29:08.941658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:29:08.941658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:08.941658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:08.941658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:08.941658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:08.941658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:08.941658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:08.941658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:08.941658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:08.941658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:08.941658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:08.941658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:08.941658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:08.941658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:08.941658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:08.941658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:08.941658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:08.941658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:08.941658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:08.942211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:08.942211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:08.942211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.942211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:08.947250 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:08.947602 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:08.947738 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:08.947883 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:08.949404 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:08.949939 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:08.950194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.951128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:08.951128 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:08.951128 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:08.951128 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:08.951128 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:08.951128 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:08.951128 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.951128 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:08.952461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.952461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:08.952461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:08.952461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:08.952461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.952461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:29:08.952461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:08.952461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:08.952461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:08.952461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:08.952461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:08.952461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:08.952882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:08.952882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:08.952882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:08.952882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:08.952882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:08.952882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:08.952882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:08.952882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:08.952882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:08.952882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:08.952882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:08.952882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:08.952882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:08.953262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.953262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:08.954015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.954015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:08.954015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:08.954015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:08.954015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:08.954015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:08.954015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.954015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:08.954015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:08.954015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.954015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:08.954015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:08.954015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.954015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:08.954015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:08.954015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:08.954520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:29:08.954520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:08.954520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:08.954520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:08.954520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:08.954520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:08.954520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:08.954520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:08.954520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:08.954520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:08.954520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:08.954520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:08.954520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:08.954520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:08.954520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:08.954520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:08.954520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:08.954520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:08.954520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:08.955062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:08.955062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:08.955062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:13.635858 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:13.637084 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:13.637269 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:13.638519 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:13.673932 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:13.673932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.673932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:13.673932 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:13.673932 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:13.673932 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:13.673932 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:13.673932 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:13.673932 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.673932 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:13.674603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.674603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:13.674603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:13.674603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:13.674603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.674603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:29:13.674603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:13.674603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:13.674603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:13.674603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:13.674603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:13.674603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:13.675349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:13.675349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:13.675349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:13.675349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:13.675349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:13.675349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:13.675349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:13.675349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:13.675349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:13.675349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:13.675349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:13.675349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:13.675349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:13.676156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.676156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:13.676156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.676156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:13.676156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:13.676156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:13.676156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:13.676156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:13.676156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.676156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:13.676156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:13.676156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.676156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:13.676156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:13.676156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.676883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:13.676883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:13.676883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.676883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:29:13.676883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:13.676883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:13.676883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:13.676883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:13.676883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:13.676883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:13.676883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:13.676883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:13.676883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:13.676883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:13.676883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:13.676883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:13.676883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:13.676883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:13.676883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:13.677494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:13.677494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:13.677494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:13.677494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:13.677494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.677494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:13.871880 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:13.872422 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:13.872537 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:13.873790 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:13.873995 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:13.874109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.874962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:13.874962 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:13.874962 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:13.874962 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:13.874962 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:13.874962 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:13.874962 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.874962 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:13.877139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.877139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:13.877139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:13.877139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:13.877139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.877139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:29:13.877139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:13.877139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:13.877139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:13.877139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:13.877139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:13.877139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:13.877836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:13.877836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:13.877836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:13.877836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:13.877836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:13.877836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:13.877836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:13.877836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:13.877836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:13.877836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:13.877836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:13.877836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:13.877836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:13.878549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.878549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:13.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:13.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:13.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:13.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:13.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:13.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:13.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:13.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:13.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:13.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:13.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:13.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.880800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:29:13.880800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:13.880800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:13.880800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:13.880800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:13.880800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:13.880800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:13.880800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:13.880800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:13.880800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:13.880800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:13.880800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:13.880800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:13.880800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:13.880800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:13.880800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:13.880800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:13.880800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:13.880800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:13.881848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:13.881848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.881848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:13.924659 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:13.925162 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:13.925328 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:13.925508 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:29:13.926278 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:13.926645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.927560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:13.927560 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:13.927560 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:13.927560 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:13.927560 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:13.927560 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:13.927560 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.927560 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:13.928514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.928514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:29:13.928514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:13.928514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:13.928514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:13.928514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:13.928514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:13.928514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:13.928514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:13.928514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:13.928514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:13.928514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:29:13.928514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:13.928514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:29:13.928514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:29:13.929317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:13.929317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:29:13.929317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:13.929317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:29:13.929317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:29:13.929317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:13.929317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:29:13.929317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.929317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:29:13.929883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.929883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:13.929883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:13.929883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:13.929883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.929883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:29:13.929883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:29:13.929883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:29:13.929883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:13.929883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:13.929883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:13.929883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:13.930494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:13.930494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:13.930494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:13.930494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:13.930494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:13.930494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:13.930494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:13.930494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:13.930494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:13.930494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:13.930494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:13.930494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:13.930494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:13.930494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:13.931125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:13.931125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:13.931125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.931125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:13.931125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.931125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:13.931125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:13.931125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:13.931125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:13.931125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:13.931125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.931125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:13.931125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:13.931125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.931756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:13.931756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:13.931756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.931756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:13.931756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:13.931756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.931756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:29:13.931756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:29:13.931756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:29:13.931756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:13.931756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:13.931756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:13.931756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:13.931756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:13.931756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:13.931756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:13.931756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:13.931756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:13.931756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:13.932287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:13.932287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:13.932287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:13.932287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:13.932287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:13.932287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:13.932287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:13.932287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:13.932287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:13.932287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:13.932287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.932287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:13.936368 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:13.936740 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:13.936873 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:13.937004 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:13.937350 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:13.937607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.938299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:13.938299 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:13.938299 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:13.938299 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:13.938299 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:13.938299 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:13.938299 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.938299 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:13.939084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.939084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:29:13.939084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:13.939084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:13.939084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:13.939084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:13.939084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:13.939084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:13.939084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:13.939084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:13.939084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:13.939084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:29:13.939084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:13.939084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:29:13.939084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:29:13.939506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:13.939506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:29:13.939506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:13.939506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:29:13.939506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:29:13.939506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:13.939506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:29:13.939506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.939506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:29:13.940031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.940031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:13.940031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:13.940031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:13.940031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.940031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:29:13.940031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:29:13.940031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:13.940031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:13.940031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:13.940031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:13.940359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:13.940359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:13.940359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:13.940359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:13.940359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:13.940359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:13.940359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:13.940359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:13.940359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:13.940359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:13.940359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:13.940359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:13.940359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:13.940359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:13.940755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:13.940755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:13.940755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.940755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:13.941123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.941123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:13.941123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:13.941123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:13.941123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:13.941123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:13.941123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.941123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:13.941123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:13.941123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.941123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:13.941123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:13.941123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.941123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:13.941123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:13.941123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.941123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:29:13.941591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:29:13.941591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:13.941591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:13.941591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:13.941591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:13.941591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:13.941591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:13.941591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:13.941591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:13.941591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:13.941591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:13.941591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:13.941591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:13.941591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:13.941591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:13.941591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:13.941591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:13.941591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:13.941591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:13.942098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:13.942098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:13.942098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.942098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:13.947795 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:13.948082 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:13.948228 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:13.948388 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:13.948904 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:13.949313 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:13.949573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.950361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:13.950361 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:13.950361 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:13.950361 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:13.950361 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:13.950361 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:13.950361 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.950361 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:13.951406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.951406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:13.951406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:13.951406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:13.951406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.951406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:29:13.951406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:13.951406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:13.951406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:13.951406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:13.951406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:13.951406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:13.952032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:13.952032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:13.952032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:13.952032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:13.952032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:13.952032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:13.952032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:13.952032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:13.952032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:13.952032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:13.952032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:13.952032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:13.952032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:13.952621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.952621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:13.952621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.952621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:13.952621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:13.952621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:13.952621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:13.952621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:13.952621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.952621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:13.952621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:13.952621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.952621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:13.952621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:13.952621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.953452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:13.953452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:13.953452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:13.953452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:29:13.953452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:13.953452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:13.953452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:13.953452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:13.953452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:13.953452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:13.953452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:13.953452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:13.953452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:13.953452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:13.953452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:13.953452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:13.953452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:13.953452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:13.953452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:13.954392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:13.954392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:13.954392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:13.954392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:13.954392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:13.954392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:18.637293 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:18.638363 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:18.638627 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:18.641935 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:18.679220 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:18.679601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.681012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:18.681012 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:18.681012 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:18.681012 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:18.681012 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:18.681012 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:18.681012 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.681012 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:18.681998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.681998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:18.681998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:18.681998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:18.681998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.681998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:29:18.681998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:18.681998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:18.681998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:18.681998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:18.681998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:18.681998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:18.683304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:18.683304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:18.683304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:18.683304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:18.683304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:18.683304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:18.683304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:18.683304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:18.683304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:18.683304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:18.683304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:18.683304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:18.683304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:18.684049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.684049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:18.684049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.684049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:18.684049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:18.684049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:18.684049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:18.684049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:18.684049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.684049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:18.684049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:18.684049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.684049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:18.684049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:18.684049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.684786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:18.684786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:18.684786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.684786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:29:18.684786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:18.684786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:18.684786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:18.684786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:18.684786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:18.684786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:18.684786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:18.684786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:18.684786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:18.684786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:18.684786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:18.684786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:18.684786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:18.684786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:18.684786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:18.685909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:18.685909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:18.685909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:18.685909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:18.685909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.685909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:18.872568 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:18.872995 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:18.873128 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:18.874511 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:18.874957 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:18.875238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.876401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:18.876401 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:18.876401 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:18.876401 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:18.876401 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:18.876401 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:18.876401 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.876401 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:18.877626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.877626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:18.877626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:18.877626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:18.877626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.877626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:29:18.877626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:18.877626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:18.877626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:18.877626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:18.877626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:18.877626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:18.878386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:18.878386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:18.878386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:18.878386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:18.878386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:18.878386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:18.878386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:18.878386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:18.878386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:18.878386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:18.878386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:18.878386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:18.878386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:18.879081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.879081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:18.879081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.879081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:18.879081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:18.879081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:18.879081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:18.879081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:18.879081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.879081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:18.879081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:18.879081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.879081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:18.879081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:18.879081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.879892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:18.879892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:18.879892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.879892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:29:18.879892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:18.879892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:18.879892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:18.879892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:18.879892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:18.879892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:18.879892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:18.879892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:18.879892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:18.879892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:18.879892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:18.879892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:18.879892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:18.879892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:18.879892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:18.880888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:18.880888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:18.880888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:18.880888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:18.880888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.880888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:18.925332 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:18.925910 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:18.926114 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:18.926339 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:29:18.927101 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:18.927441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.928349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:18.928349 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:18.928349 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:18.928349 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:18.928349 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:18.928349 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:18.928349 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.928349 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:18.929668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.929668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:29:18.929668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:18.929668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:18.929668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:18.929668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:18.929668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:18.929668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:18.929668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:18.929668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:18.929668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:18.929668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:29:18.929668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:18.929668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:29:18.929668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:29:18.930100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:18.930100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:29:18.930100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:18.930100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:29:18.930100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:29:18.930100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:18.930100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:29:18.930100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.930100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:29:18.931144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.931144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:18.931144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:18.931144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:18.931144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.931144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:29:18.931144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:29:18.931144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:29:18.931144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:18.931144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:18.931144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:18.931144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:18.931613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:18.931613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:18.931613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:18.931613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:18.931613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:18.931613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:18.931613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:18.931613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:18.931613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:18.931613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:18.931613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:18.931613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:18.931613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:18.931613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:18.931958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:18.931958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:18.931958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.931958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:18.932981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.932981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:18.932981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:18.932981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:18.932981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:18.932981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:18.932981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.932981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:18.932981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:18.932981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.932981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:18.932981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:18.932981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.932981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:18.932981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:18.932981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.933347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:29:18.933347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:29:18.933347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:29:18.933347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:18.933347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:18.933347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:18.933347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:18.933347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:18.933347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:18.933347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:18.933347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:18.933347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:18.933347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:18.933347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:18.933347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:18.933347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:18.933347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:18.933347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:18.933815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:18.933815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:18.933815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:18.933815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:18.933815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:18.933815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.933815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:18.936874 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:18.937179 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:18.937301 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:18.937473 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:18.937981 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:18.938340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.939025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:18.939025 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:18.939025 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:18.939025 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:18.939025 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:18.939025 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:18.939025 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.939025 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:18.939765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.939765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:29:18.939765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:18.939765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:18.939765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:18.939765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:18.939765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:18.939765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:18.939765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:18.939765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:18.939765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:18.939765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:29:18.939765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:18.939765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:29:18.939765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:29:18.940093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:18.940093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:29:18.940093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:18.940093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:29:18.940093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:29:18.940093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:18.940093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:29:18.940093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.940093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:29:18.940607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.940607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:18.940607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:18.940607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:18.940607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.940607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:29:18.940607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:29:18.940607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:18.940607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:18.940607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:18.940607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:18.940904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:18.940904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:18.940904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:18.940904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:18.940904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:18.940904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:18.940904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:18.940904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:18.940904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:18.940904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:18.940904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:18.940904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:18.940904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:18.940904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:18.941209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:18.941209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:18.941209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.941209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:18.941741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.941741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:18.941741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:18.941741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:18.941741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:18.941741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:18.941741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.941741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:18.941741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:18.941741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.941741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:18.941741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:18.941741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.941741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:18.941741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:18.941741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.941741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:29:18.942133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:29:18.942133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:18.942133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:18.942133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:18.942133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:18.942133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:18.942133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:18.942133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:18.942133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:18.942133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:18.942133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:18.942133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:18.942133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:18.942133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:18.942133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:18.942133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:18.942133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:18.942133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:18.942133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:18.942552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:18.942552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:18.942552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.942552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:18.948268 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:18.948589 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:18.948710 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:18.948888 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:18.949332 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:18.949813 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:18.950049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.950736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:18.950736 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:18.950736 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:18.950736 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:18.950736 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:18.950736 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:18.950736 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.950736 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:18.952011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.952011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:18.952011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:18.952011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:18.952011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.952011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:29:18.952011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:18.952011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:18.952011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:18.952011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:18.952011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:18.952011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:18.952325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:18.952325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:18.952325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:18.952325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:18.952325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:18.952325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:18.952325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:18.952325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:18.952325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:18.952325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:18.952325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:18.952325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:18.952325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:18.952604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.952604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:18.953827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.953827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:18.953827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:18.953827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:18.953827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:18.953827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:18.953827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.953827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:18.953827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:18.953827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.953827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:18.953827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:18.953827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.953827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:18.953827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:18.953827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:18.954203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:29:18.954203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:18.954203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:18.954203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:18.954203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:18.954203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:18.954203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:18.954203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:18.954203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:18.954203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:18.954203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:18.954203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:18.954203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:18.954203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:18.954203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:18.954203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:18.954203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:18.954203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:18.954203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:18.954625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:18.954625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:18.954625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:23.638232 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:23.639167 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:23.639167 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:23.639780 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:23.674601 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:23.675077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.675963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:23.675963 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:23.675963 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:23.675963 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:23.675963 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:23.675963 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:23.675963 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.675963 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:23.676946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.676946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:23.676946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:23.676946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:23.676946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.676946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:29:23.676946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:23.676946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:23.676946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:23.676946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:23.676946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:23.676946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:23.677612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:23.677612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:23.677612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:23.677612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:23.677612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:23.677612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:23.677612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:23.677612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:23.677612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:23.677612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:23.677612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:23.677612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:23.677612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:23.679015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.679015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:23.679015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.679015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:23.679015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:23.679015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:23.679015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:23.679015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:23.679015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.679015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:23.679015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:23.679015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.679015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:23.679015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:23.679015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.680147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:23.680147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:23.680147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.680147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:29:23.680147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:23.680147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:23.680147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:23.680147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:23.680147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:23.680147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:23.680147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:23.680147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:23.680147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:23.680147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:23.680147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:23.680147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:23.680147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:23.680147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:23.680147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:23.682067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:23.682067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:23.682067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:23.682067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:23.682067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.682067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:23.873835 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:23.873835 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:23.874142 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:23.874855 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:23.874994 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:23.875087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.875639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:23.875639 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:23.875639 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:23.875639 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:23.875639 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:23.875639 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:23.875639 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.875639 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:23.877246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.877246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:23.877246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:23.877246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:23.877246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.877246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:29:23.877246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:23.877246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:23.877246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:23.877246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:23.877246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:23.877246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:23.878257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:23.878257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:23.878257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:23.878257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:23.878257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:23.878257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:23.878257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:23.878257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:23.878257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:23.878257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:23.878257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:23.878257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:23.878257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:23.878802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.878802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:23.878895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.878895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:23.878895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:23.878895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:23.878895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:23.878895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:23.878895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.878895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:23.878895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:23.878895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.878895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:23.878895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:23.878895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.878895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:23.878895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:23.878895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.880117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:29:23.880117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:23.880117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:23.880117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:23.880117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:23.880117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:23.880117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:23.880117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:23.880117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:23.880117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:23.880117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:23.880117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:23.880117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:23.880117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:23.880117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:23.880117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:23.880117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:23.880117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:23.880117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:23.880868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:23.880868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.880868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:23.925969 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:23.926536 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:23.926747 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:23.926939 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:29:23.927524 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:23.927879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.928645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:23.928645 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:23.928645 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:23.928645 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:23.928645 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:23.928645 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:23.928645 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.928645 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:23.929852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.929852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:29:23.929852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:23.929852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:23.929852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:23.929852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:23.929852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:23.929852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:23.929852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:23.929852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:23.929852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:23.929852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:29:23.929852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:23.929852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:29:23.929852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:29:23.930575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:23.930575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:29:23.930575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:23.930575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:29:23.930575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:29:23.930575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:23.930575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:29:23.930575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.930575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:29:23.931224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.931224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:23.931224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:23.931224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:23.931224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.931224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:29:23.931224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:29:23.931224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:29:23.931224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:23.931224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:23.931224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:23.931224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:23.931795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:23.931795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:23.931795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:23.931795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:23.931795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:23.931795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:23.931795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:23.931795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:23.931795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:23.931795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:23.931795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:23.931795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:23.931795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:23.931795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:23.932391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:23.932391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:23.932391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.932391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:23.932925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.932925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:23.932925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:23.932925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:23.932925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:23.932925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:23.932925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.932925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:23.932925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:23.932925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.932925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:23.932925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:23.932925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.932925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:23.932925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:23.932925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.933663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:29:23.933663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:29:23.933663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:29:23.933663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:23.933663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:23.933663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:23.933663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:23.933663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:23.933663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:23.933663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:23.933663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:23.933663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:23.933663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:23.933663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:23.933663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:23.933663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:23.933663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:23.933663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:23.934468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:23.934468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:23.934468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:23.934468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:23.934468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:23.934468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.934468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:23.937335 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:23.937640 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:23.939385 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:23.939385 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:23.939833 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:23.940196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.941827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:23.941827 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:23.941827 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:23.941827 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:23.941827 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:23.941827 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:23.941827 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.941827 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:23.941827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.941827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:29:23.941827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:23.941827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:23.941827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:23.941827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:23.941827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:23.941827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:23.941827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:23.941827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:23.942812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:23.942812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:29:23.942812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:23.942812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:29:23.942812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:29:23.942812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:23.942812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:29:23.942812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:23.942812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:29:23.942812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:29:23.942812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:23.942812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:29:23.942812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.942812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:29:23.942812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.943566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:23.943566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:23.943566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:23.943566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.943566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:29:23.943566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:29:23.943566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:23.943566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:23.943566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:23.943566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:23.943566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:23.943566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:23.943566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:23.944226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:23.944226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:23.944226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:23.944226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:23.944226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:23.944226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:23.944226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:23.944226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:23.944226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:23.944226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:23.944226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:23.944226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:23.944226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:23.944867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.944867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:23.944867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.944867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:23.944867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:23.944867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:23.944867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:23.944867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:23.944867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.944867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:23.944867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:23.944867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.944867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:23.944867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:23.944867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.945592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:23.945592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:23.945592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.945592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:29:23.945592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:29:23.945592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:23.945592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:23.945592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:23.945592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:23.945592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:23.945592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:23.945592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:23.945592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:23.945592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:23.945592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:23.945592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:23.945592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:23.945592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:23.945592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:23.946553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:23.946553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:23.946553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:23.946553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:23.946553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:23.946553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:23.946553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.946553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:23.948729 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:23.949033 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:23.949186 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:23.949528 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:23.949916 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:23.950299 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:23.950545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.951299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:23.951299 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:23.951299 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:23.951299 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:23.951299 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:23.951299 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:23.951299 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.951299 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:23.952185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.952185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:23.952185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:23.952185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:23.952185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.952185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:29:23.952185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:23.952185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:23.952185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:23.952185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:23.952185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:23.952185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:23.952691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:23.952691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:23.952691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:23.952691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:23.952691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:23.952691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:23.952691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:23.952691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:23.952691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:23.952691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:23.952691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:23.952691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:23.952691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:23.953029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.953029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:23.953265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.953265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:23.953265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:23.953265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:23.953265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:23.953265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:23.953265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.953265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:23.953265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:23.953265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.953265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:23.953265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:23.953265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.953265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:23.953265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:23.953265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:23.953771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:29:23.953771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:23.953771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:23.953771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:23.953771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:23.953771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:23.953771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:23.953771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:23.953771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:23.953771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:23.953771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:23.953771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:23.953771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:23.953771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:23.953771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:23.953771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:23.953771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:23.953771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:23.953771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:23.955046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:23.955046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:23.955046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:28.638822 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:28.639683 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:28.639683 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:28.640395 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:28.672007 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:28.672354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.673304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:28.673304 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:28.673304 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:28.673304 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:28.673304 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:28.673304 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:28.673304 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.673304 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:28.674492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.674492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:28.674492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:28.674492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:28.674492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.674492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:29:28.674492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:28.674492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:28.674492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:28.674492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:28.674492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:28.674492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:28.674857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:28.674857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:28.674857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:28.674857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:28.674857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:28.674857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:28.674857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:28.674857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:28.674857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:28.674857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:28.674857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:28.674857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:28.674857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:28.675181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.675181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:28.675573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.675573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:28.675573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:28.675573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:28.675573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:28.675573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:28.675573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.675573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:28.675573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:28.675573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.675573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:28.675573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:28.675573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.675573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:28.675573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:28.675573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.676602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:29:28.676602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:28.676602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:28.676602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:28.676602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:28.676602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:28.676602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:28.676602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:28.676602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:28.676602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:28.676602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:28.676602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:28.676602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:28.676602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:28.676602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:28.676602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:28.676602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:28.676602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:28.676602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:28.677916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:28.677916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.677916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:28.874003 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:28.875194 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:28.875194 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:28.876097 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:28.876412 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:28.876767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.877628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:28.877628 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:28.877628 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:28.877628 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:28.877628 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:28.877628 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:28.877628 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.877628 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:28.879246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.879246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:28.879246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:28.879246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:28.879246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.879246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:29:28.879246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:28.879246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:28.879246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:28.879246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:28.879246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:28.879246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:28.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:28.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:28.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:28.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:28.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:28.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:28.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:28.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:28.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:28.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:28.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:28.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:28.879951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:28.880917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.880917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:28.881146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.881146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:28.881146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:28.881146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:28.881146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:28.881146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:28.881146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.881146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:28.881146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:28.881146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.881146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:28.881146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:28.881146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.881146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:28.881146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:28.881146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.882553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:29:28.882553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:28.882553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:28.882553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:28.882553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:28.882553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:28.882553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:28.882553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:28.882553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:28.882553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:28.882553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:28.882553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:28.882553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:28.882553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:28.882553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:28.882553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:28.882553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:28.882553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:28.882553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:28.883466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:28.883466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.883466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:28.926638 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:28.927161 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:28.927376 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:28.927560 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:29:28.928169 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:28.928473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.929407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:28.929407 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:28.929407 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:28.929407 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:28.929407 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:28.929407 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:28.929407 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.929407 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:28.930544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.930544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:29:28.930544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:28.930544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:28.930544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:28.930544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:28.930544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:28.930544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:28.930544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:28.930544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:28.930544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:28.930544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:29:28.930544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:28.930544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:29:28.930544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:29:28.931188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:28.931188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:29:28.931188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:28.931188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:29:28.931188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:29:28.931188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:28.931188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:29:28.931188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.931188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:29:28.931885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.931885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:28.931885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:28.931885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:28.931885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.931885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:29:28.931885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:29:28.931885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:29:28.931885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:28.931885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:28.931885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:28.931885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:28.932398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:28.932398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:28.932398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:28.932398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:28.932398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:28.932398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:28.932398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:28.932398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:28.932398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:28.932398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:28.932398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:28.932398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:28.932398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:28.932398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:28.933029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:28.933029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:28.933029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.933029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:28.933554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.933554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:28.933554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:28.933554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:28.933554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:28.933554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:28.933554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.933554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:28.933554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:28.933554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.933554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:28.933554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:28.933554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.933554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:28.933554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:28.933554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.934241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:29:28.934241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:29:28.934241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:29:28.934241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:28.934241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:28.934241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:28.934241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:28.934241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:28.934241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:28.934241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:28.934241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:28.934241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:28.934241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:28.934241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:28.934241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:28.934241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:28.934241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:28.934241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:28.935219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:28.935219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:28.935219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:28.935219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:28.935219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:28.935219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.935219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:28.937880 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:28.938347 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:28.938551 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:28.938806 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:28.939166 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:28.939399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.939952 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:28.941331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.941331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:29:28.941331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:28.941331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:28.941331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:28.941331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:28.941331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:28.941331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:28.941331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:28.941331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:28.941331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:28.941331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:29:28.941331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:28.941331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:29:28.941331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:29:28.941761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:28.941761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:29:28.941761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:28.941761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:29:28.941761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:29:28.941761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:28.941761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:29:28.941761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.941761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:29:28.942914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.942914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:28.942914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:28.942914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:28.942914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.942914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:29:28.942914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:29:28.942914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:28.942914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:28.942914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:28.942914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:28.943209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:28.943209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:28.943209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:28.943209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:28.943209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:28.943209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:28.943209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:28.943209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:28.943209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:28.943209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:28.943209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:28.943209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:28.943209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:28.943209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:28.943563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:28.943563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:28.943563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.943563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:28.944843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.944843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:28.944843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:28.944843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:28.944843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:28.944843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:28.944843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.944843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:28.944843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:28.944843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.944843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:28.944843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:28.944843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.944843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:28.944843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:28.944843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.944843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:29:28.945401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:29:28.945401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:28.945401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:28.945401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:28.945401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:28.945401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:28.945401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:28.945401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:28.945401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:28.945401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:28.945401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:28.945401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:28.945401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:28.945401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:28.945401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:28.945401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:28.945401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:28.945401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:28.945401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:28.945949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:28.945949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:28.945949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.945949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:28.949250 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:28.949464 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:28.949541 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:28.949655 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:28.949999 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:28.950230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:28.950409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.950898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:28.950898 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:28.950898 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:28.950898 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:28.950898 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:28.950898 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:28.950898 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.950898 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:28.951456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.951456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:28.951456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:28.951456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:28.951456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.951456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:29:28.951456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:28.951456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:28.951456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:28.951456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:28.951456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:28.951456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:28.951820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:28.951820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:28.951820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:28.951820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:28.951820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:28.951820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:28.951820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:28.951820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:28.951820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:28.951820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:28.951820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:28.951820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:28.951820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:28.952199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.952199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:28.952199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.952199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:28.952199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:28.952199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:28.952199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:28.952199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:28.952199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.952199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:28.952199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:28.952199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.952199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:28.952199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:28.952199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.952612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:28.952612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:28.952612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:28.952612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:29:28.952612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:28.952612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:28.952612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:28.952612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:28.952612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:28.952612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:28.952612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:28.952612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:28.952612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:28.952612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:28.952612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:28.952612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:28.952612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:28.952612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:28.952612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:28.953137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:28.953137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:28.953137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:28.953137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:28.953137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:28.953137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:33.639556 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:33.640303 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:33.640390 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:33.641707 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:33.678822 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:33.679220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.680201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:33.680201 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:33.680201 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:33.680201 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:33.680201 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:33.680201 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:33.680201 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.680201 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:33.681275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.681275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:33.681275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:33.681275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:33.681275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.681275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:29:33.681275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:33.681275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:33.681275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:33.681275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:33.681275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:33.681275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:33.682027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:33.682027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:33.682027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:33.682027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:33.682027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:33.682027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:33.682027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:33.682027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:33.682027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:33.682027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:33.682027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:33.682027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:33.682027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:33.682702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.682702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:33.682702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.682702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:33.682702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:33.682702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:33.682702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:33.682702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:33.682702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.682702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:33.682702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:33.682702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.682702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:33.682702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:33.682702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.683491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:33.683491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:33.683491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.683491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:29:33.683491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:33.683491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:33.683491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:33.683491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:33.683491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:33.683491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:33.683491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:33.683491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:33.683491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:33.683491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:33.683491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:33.683491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:33.683491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:33.683491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:33.683491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:33.684391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:33.684391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:33.684391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:33.684391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:33.684391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.684391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:33.874986 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:33.875482 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:33.875704 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:33.876952 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:33.877429 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:33.877725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.878797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:33.878797 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:33.878797 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:33.878797 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:33.878797 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:33.878797 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:33.878797 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.878797 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:33.880414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.880414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:33.880414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:33.880414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:33.880414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.880414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:29:33.880414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:33.880414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:33.880414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:33.880414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:33.880414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:33.880414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:33.881176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:33.881176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:33.881176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:33.881176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:33.881176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:33.881176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:33.881176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:33.881176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:33.881176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:33.881176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:33.881176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:33.881176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:33.881176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:33.881543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.881543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:33.882315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.882315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:33.882315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:33.882315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:33.882315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:33.882315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:33.882315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.882315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:33.882315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:33.882315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.882315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:33.882315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:33.882315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.882315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:33.882315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:33.882315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.882810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:29:33.882810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:33.882810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:33.882810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:33.882810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:33.882810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:33.882810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:33.882810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:33.882810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:33.882810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:33.882810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:33.882810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:33.882810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:33.882810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:33.882810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:33.882810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:33.882810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:33.882810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:33.882810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:33.883304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:33.883304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.883304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:33.927291 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:33.927914 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:33.928069 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:33.928279 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:29:33.929035 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:33.929398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.930452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:33.930452 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:33.930452 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:33.930452 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:33.930452 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:33.930452 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:33.930452 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.930452 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:33.931667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.931667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:29:33.931667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:33.931667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:33.931667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:33.931667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:33.931667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:33.931667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:33.931667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:33.931667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:33.931667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:33.931667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:29:33.931667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:33.931667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:29:33.931667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:29:33.932518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:33.932518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:29:33.932518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:33.932518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:29:33.932518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:29:33.932518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:33.932518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:29:33.932518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.932518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:29:33.933133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.933133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:33.933133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:33.933133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:33.933133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.933133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:29:33.933133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:29:33.933133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:29:33.933133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:33.933133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:33.933133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:33.933133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:33.933882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:33.933882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:33.933882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:33.933882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:33.933882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:33.933882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:33.933882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:33.933882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:33.933882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:33.933882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:33.933882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:33.933882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:33.933882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:33.933882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:33.934442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:33.934442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:33.934442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.934442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:33.934819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.934819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:33.934819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:33.934819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:33.934819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:33.934819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:33.934819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.934819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:33.934819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:33.934819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.934819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:33.934819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:33.934819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.934819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:33.934819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:33.934819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.935285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:29:33.935285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:29:33.935285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:29:33.935285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:33.935285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:33.935285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:33.935285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:33.935285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:33.935285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:33.935285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:33.935285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:33.935285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:33.935285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:33.935285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:33.935285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:33.935285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:33.935285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:33.935285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:33.935840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:33.935840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:33.935840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:33.935840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:33.935840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:33.935840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.935840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:33.938405 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:33.938802 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:33.938994 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:33.939207 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:33.939550 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:33.939828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.940304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:33.940304 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:33.940304 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:33.940304 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:33.940304 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:33.940304 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:33.940304 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.940304 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:33.941019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.941019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:29:33.941019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:33.941019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:33.941019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:33.941019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:33.941019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:33.941019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:33.941019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:33.941019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:33.941019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:33.941019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:29:33.941019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:33.941019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:29:33.941019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:29:33.941628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:33.941628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:29:33.941628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:33.941628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:29:33.941628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:29:33.941628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:33.941628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:29:33.941628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.941628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:29:33.942111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.942111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:33.942111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:33.942111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:33.942111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.942111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:29:33.942111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:29:33.942111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:33.942111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:33.942111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:33.942111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:33.942527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:33.942527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:33.942527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:33.942527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:33.942527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:33.942527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:33.942527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:33.942527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:33.942527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:33.942527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:33.942527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:33.942527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:33.942527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:33.942527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:33.942931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:33.942931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:33.942931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.942931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:33.942931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.942931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:33.942931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:33.942931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:33.942931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:33.942931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:33.942931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.942931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:33.942931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:33.942931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.943320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:33.943320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:33.943320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.943320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:33.943320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:33.943320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.943320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:29:33.943320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:29:33.943320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:33.943320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:33.943320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:33.943320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:33.943320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:33.943320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:33.943320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:33.943320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:33.943320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:33.943320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:33.943320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:33.944105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:33.944105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:33.944105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:33.944105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:33.944105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:33.944105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:33.944105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:33.944105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:33.944105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:33.944105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.944105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:33.949740 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:33.950068 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:33.950182 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:33.950464 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:33.950790 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:33.951054 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:33.951230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.951894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:33.951894 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:33.951894 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:33.951894 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:33.951894 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:33.951894 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:33.951894 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.951894 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:33.953235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.953235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:33.953235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:33.953235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:33.953235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.953235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:29:33.953235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:33.953235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:33.953235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:33.953235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:33.953235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:33.953235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:33.953632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:33.953632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:33.953632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:33.953632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:33.953632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:33.953632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:33.953632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:33.953632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:33.953632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:33.953632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:33.953632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:33.953632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:33.953632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:33.954136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.954136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:33.955001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.955001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:33.955001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:33.955001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:33.955001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:33.955001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:33.955001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.955001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:33.955001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:33.955001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.955001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:33.955001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:33.955001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.955001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:33.955001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:33.955001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:33.955554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:29:33.955554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:33.955554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:33.955554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:33.955554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:33.955554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:33.955554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:33.955554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:33.955554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:33.955554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:33.955554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:33.955554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:33.955554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:33.955554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:33.955554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:33.955554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:33.955554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:33.955554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:33.955554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:33.956141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:33.956141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:33.956141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:38.640323 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:38.641303 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:38.641303 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:38.641741 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:38.680355 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:38.680826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.681636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:38.681636 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:38.681636 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:38.681636 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:38.681636 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:38.681636 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:38.681636 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.681636 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:38.682656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.682656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:38.682656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:38.682656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:38.682656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.682656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:29:38.682656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:38.682656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:38.682656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:38.682656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:38.682656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:38.682656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:38.683381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:38.683381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:38.683381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:38.683381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:38.683381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:38.683381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:38.683381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:38.683381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:38.683381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:38.683381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:38.683381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:38.683381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:38.683381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:38.684060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.684060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:38.684060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.684060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:38.684060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:38.684060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:38.684060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:38.684060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:38.684060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.684060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:38.684060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:38.684060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.684060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:38.684060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:38.684060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.684866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:38.684866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:38.684866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.684866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:29:38.684866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:38.684866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:38.684866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:38.684866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:38.684866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:38.684866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:38.684866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:38.684866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:38.684866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:38.684866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:38.684866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:38.684866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:38.684866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:38.684866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:38.684866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:38.686417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:38.686417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:38.686417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:38.686417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:38.686417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.686417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:38.875512 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:38.876083 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:38.876280 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:38.877231 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:38.877537 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:38.877820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.879064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:38.879064 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:38.879064 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:38.879064 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:38.879064 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:38.879064 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:38.879064 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.879064 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:38.880220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.880220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:38.880220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:38.880220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:38.880220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.880220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:29:38.880220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:38.880220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:38.880220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:38.880220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:38.880220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:38.880220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:38.880782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:38.880782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:38.880782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:38.880782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:38.880782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:38.880782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:38.880782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:38.880782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:38.880782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:38.880782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:38.880782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:38.880782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:38.880782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:38.881516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.881516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:38.883366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.883366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:38.883366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:38.883366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:38.883366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:38.883366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:38.883366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.883366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:38.883366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:38.883366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.883366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:38.883366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:38.883366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.883366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:38.883366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:38.883366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.884412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:29:38.884412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:38.884412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:38.884412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:38.884412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:38.884412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:38.884412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:38.884412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:38.884412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:38.884412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:38.884412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:38.884412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:38.884412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:38.884412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:38.884412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:38.884412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:38.884412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:38.884412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:38.884412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:38.884897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:38.884897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.884897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:38.928151 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:38.928574 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:38.928667 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:38.928851 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:29:38.929356 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:38.929660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.930585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:38.930585 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:38.930585 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:38.930585 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:38.930585 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:38.930585 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:38.930585 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.930585 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:38.931862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.931862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:29:38.931862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:38.931862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:38.931862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:38.931862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:38.931862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:38.931862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:38.931862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:38.931862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:38.931862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:38.931862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:29:38.931862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:38.931862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:29:38.931862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:29:38.932616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:38.932616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:29:38.932616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:38.932616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:29:38.932616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:29:38.932616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:38.932616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:29:38.932616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.932616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:29:38.933230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.933230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:38.933230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:38.933230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:38.933230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.933230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:29:38.933230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:29:38.933230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:29:38.933230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:38.933230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:38.933230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:38.933230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:38.933817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:38.933817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:38.933817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:38.933817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:38.933817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:38.933817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:38.933817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:38.933817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:38.933817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:38.933817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:38.933817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:38.933817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:38.933817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:38.933817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:38.934469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:38.934469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:38.934469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.934469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:38.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:38.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:38.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:38.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:38.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:38.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:38.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:38.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:38.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:38.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:38.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:38.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.935828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:29:38.935828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:29:38.935828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:29:38.935828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:38.935828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:38.935828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:38.935828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:38.935828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:38.935828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:38.935828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:38.935828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:38.935828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:38.935828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:38.935828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:38.935828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:38.935828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:38.935828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:38.935828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:38.936806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:38.936806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:38.936806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:38.936806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:38.936806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:38.936806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.936806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:38.938845 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:38.939244 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:38.939400 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:38.939519 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:38.939874 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:38.940099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.940529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:38.940529 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:38.940529 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:38.940529 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:38.940529 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:38.940529 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:38.940529 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.940529 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:38.941230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.941230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:29:38.941230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:38.941230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:38.941230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:38.941230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:38.941230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:38.941230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:38.941230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:38.941230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:38.941230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:38.941230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:29:38.941230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:38.941230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:29:38.941230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:29:38.941696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:38.941696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:29:38.941696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:38.941696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:29:38.941696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:29:38.941696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:38.941696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:29:38.941696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.941696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:29:38.942067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.942067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:38.942067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:38.942067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:38.942067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.942067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:29:38.942067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:29:38.942067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:38.942067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:38.942067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:38.942067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:38.942451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:38.942451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:38.942451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:38.942451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:38.942451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:38.942451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:38.942451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:38.942451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:38.942451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:38.942451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:38.942451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:38.942451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:38.942451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:38.942451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:38.942859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:38.942859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:38.942859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.942859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:38.943235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.943235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:38.943235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:38.943235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:38.943235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:38.943235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:38.943235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.943235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:38.943235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:38.943235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.943235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:38.943235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:38.943235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.943235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:38.943235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:38.943235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.943235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:29:38.943774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:29:38.943774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:38.943774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:38.943774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:38.943774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:38.943774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:38.943774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:38.943774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:38.943774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:38.943774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:38.943774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:38.943774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:38.943774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:38.943774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:38.943774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:38.943774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:38.943774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:38.943774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:38.943774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:38.944245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:38.944245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:38.944245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.944245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:38.950197 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:38.950495 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:38.950602 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:38.950839 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:38.951119 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:38.951341 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:38.951472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.951945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:38.951945 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:38.951945 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:38.951945 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:38.951945 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:38.951945 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:38.951945 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.951945 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:38.952701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.952701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:38.952701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:38.952701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:38.952701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.952701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:29:38.952701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:38.952701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:38.952701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:38.952701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:38.952701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:38.952701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:38.953105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:38.953105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:38.953105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:38.953105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:38.953105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:38.953105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:38.953105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:38.953105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:38.953105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:38.953105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:38.953105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:38.953105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:38.953105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:38.953470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.953470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:38.953809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.953809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:38.953809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:38.953809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:38.953809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:38.953809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:38.953809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.953809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:38.953809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:38.953809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.953809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:38.953809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:38.953809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.953809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:38.953809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:38.953809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:38.954342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:29:38.954342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:38.954342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:38.954342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:38.954342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:38.954342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:38.954342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:38.954342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:38.954342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:38.954342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:38.954342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:38.954342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:38.954342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:38.954342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:38.954342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:38.954342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:38.954342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:38.954342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:38.954342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:38.954863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:38.954863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:38.954863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:43.641175 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:43.642204 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:43.642204 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:43.643281 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:43.681489 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:43.681912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.682769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:43.682769 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:43.682769 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:43.682769 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:43.682769 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:43.682769 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:43.682769 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.682769 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:43.683665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.683665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:43.683665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:43.683665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:43.683665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.683665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:29:43.683665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:43.683665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:43.683665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:43.683665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:43.683665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:43.683665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:43.684405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:43.684405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:43.684405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:43.684405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:43.684405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:43.684405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:43.684405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:43.684405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:43.684405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:43.684405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:43.684405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:43.684405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:43.684405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:43.685223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.685223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:43.685223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.685223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:43.685223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:43.685223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:43.685223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:43.685223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:43.685223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.685223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:43.685223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:43.685223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.685223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:43.685223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:43.685223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.686214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:43.686214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:43.686214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.686214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:29:43.686214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:43.686214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:43.686214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:43.686214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:43.686214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:43.686214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:43.686214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:43.686214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:43.686214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:43.686214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:43.686214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:43.686214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:43.686214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:43.686214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:43.686214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:43.687307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:43.687307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:43.687307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:43.687307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:43.687307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.687307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:43.876297 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:43.877005 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:43.877005 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:43.878278 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:43.878594 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:43.878846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.880151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:43.880151 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:43.880151 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:43.880151 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:43.880151 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:43.880151 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:43.880151 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.880151 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:43.881271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.881271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:43.881271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:43.881271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:43.881271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.881271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:29:43.881271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:43.881271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:43.881271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:43.881271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:43.881271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:43.881271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:43.881799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:43.881799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:43.881799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:43.881799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:43.881799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:43.881799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:43.881799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:43.881799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:43.881799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:43.881799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:43.881799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:43.881799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:43.881799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:43.882179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.882179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:43.883295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.883295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:43.883295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:43.883295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:43.883295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:43.883295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:43.883295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.883295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:43.883295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:43.883295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.883295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:43.883295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:43.883295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.883295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:43.883295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:43.883295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.883903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:29:43.883903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:43.883903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:43.883903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:43.883903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:43.883903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:43.883903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:43.883903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:43.883903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:43.883903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:43.883903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:43.883903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:43.883903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:43.883903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:43.883903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:43.883903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:43.883903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:43.883903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:43.883903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:43.884862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:43.884862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.884862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:43.928868 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:43.929378 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:43.929471 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:43.929625 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:29:43.930250 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:43.930592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.931455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:43.931455 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:43.931455 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:43.931455 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:43.931455 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:43.931455 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:43.931455 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.931455 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:43.932357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.932357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:29:43.932357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:43.932357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:43.932357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:43.932357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:43.932357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:43.932357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:43.932357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:43.932357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:43.932357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:43.932357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:29:43.932357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:43.932357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:29:43.932357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:29:43.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:43.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:29:43.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:43.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:29:43.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:29:43.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:43.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:29:43.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:29:43.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:43.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:43.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:43.933441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.934264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:29:43.934264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:29:43.934264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:29:43.934264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:43.934264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:43.934264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:43.934264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:43.934264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:43.934264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:43.934264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:43.934264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:43.934264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:43.934264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:43.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:43.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:43.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:43.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:43.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:43.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:43.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:43.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:43.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:43.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:43.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:43.935566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.935566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:43.935566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:43.935566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:43.935566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:43.935566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:43.935566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.935566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:43.935566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:43.935566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.935566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:43.935566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:43.935566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.935566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:43.935566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:43.935566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.936398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:29:43.936398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:29:43.936398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:29:43.936398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:43.936398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:43.936398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:43.936398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:43.936398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:43.936398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:43.936398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:43.936398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:43.936398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:43.936398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:43.936398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:43.936398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:43.936398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:43.936398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:43.936398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:43.936975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:43.936975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:43.936975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:43.936975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:43.936975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:43.936975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.936975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:43.939299 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:43.939609 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:43.939751 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:43.939879 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:43.940389 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:43.940660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.941170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:43.941170 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:43.941170 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:43.941170 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:43.941170 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:43.941170 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:43.941170 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.941170 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:43.942001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.942001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:29:43.942001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:43.942001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:43.942001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:43.942001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:43.942001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:43.942001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:43.942001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:43.942001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:43.942001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:43.942001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:29:43.942001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:43.942001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:29:43.942001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:29:43.942717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:43.942717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:29:43.942717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:43.942717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:29:43.942717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:29:43.942717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:43.942717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:29:43.942717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.942717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:29:43.943165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.943165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:43.943165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:43.943165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:43.943165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.943165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:29:43.943165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:29:43.943165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:43.943165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:43.943165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:43.943165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:43.943746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:43.943746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:43.943746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:43.943746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:43.943746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:43.943746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:43.943746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:43.943746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:43.943746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:43.943746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:43.943746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:43.943746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:43.943746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:43.943746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:43.944484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:43.944484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:43.944484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.944484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:43.944484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.944484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:43.944484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:43.944484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:43.944484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:43.944484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:43.944484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.944484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:43.944484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:43.944484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.945190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:43.945190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:43.945190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.945190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:43.945190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:43.945190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.945190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:29:43.945190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:29:43.945190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:43.945190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:43.945190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:43.945190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:43.945190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:43.945190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:43.945190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:43.945190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:43.945190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:43.945190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:43.945190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:43.946106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:43.946106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:43.946106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:43.946106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:43.946106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:43.946106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:43.946106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:43.946106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:43.946106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:43.946106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.946106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:43.950770 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:43.951111 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:43.951226 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:43.951377 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:43.951664 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:43.951927 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:43.952093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.952589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:43.952589 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:43.952589 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:43.952589 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:43.952589 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:43.952589 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:43.952589 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.952589 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:43.953453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.953453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:43.953453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:43.953453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:43.953453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.953453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:29:43.953453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:43.953453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:43.953453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:43.953453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:43.953453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:43.953453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:43.953823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:43.953823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:43.953823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:43.953823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:43.953823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:43.953823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:43.953823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:43.953823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:43.953823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:43.953823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:43.953823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:43.953823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:43.953823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:43.954174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.954174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:43.954434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.954434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:43.954434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:43.954434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:43.954434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:43.954434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:43.954434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.954434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:43.954434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:43.954434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.954434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:43.954434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:43.954434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.954434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:43.954434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:43.954434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:43.954940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:29:43.954940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:43.954940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:43.954940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:43.954940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:43.954940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:43.954940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:43.954940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:43.954940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:43.954940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:43.954940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:43.954940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:43.954940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:43.954940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:43.954940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:43.954940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:43.954940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:43.954940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:43.954940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:43.955519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:43.955519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:43.955519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:48.641804 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:48.642657 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:48.642657 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:48.643809 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:48.676540 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:48.677000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.678276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:48.678276 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:48.678276 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:48.678276 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:48.678276 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:48.678276 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:48.678276 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.678276 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:48.679187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.679187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:48.679187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:48.679187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:48.679187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.679187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:29:48.679187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:48.679187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:48.679187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:48.679187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:48.679187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:48.679187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:48.679774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:48.679774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:48.679774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:48.679774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:48.679774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:48.679774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:48.679774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:48.679774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:48.679774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:48.679774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:48.679774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:48.679774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:48.679774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:48.680263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.680263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:48.680263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.680263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:48.680263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:48.680263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:48.680263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:48.680263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:48.680263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.680263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:48.680263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:48.680263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.680263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:48.680263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:48.680263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.681059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:48.681059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:48.681059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.681059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:29:48.681059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:48.681059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:48.681059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:48.681059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:48.681059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:48.681059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:48.681059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:48.681059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:48.681059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:48.681059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:48.681059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:48.681059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:48.681059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:48.681059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:48.681059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:48.681933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:48.681933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:48.681933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:48.681933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:48.681933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.681933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:48.877021 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:48.877496 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:48.877629 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:48.880646 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:48.880933 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:48.881137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.881855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:48.881855 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:48.881855 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:48.881855 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:48.881855 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:48.881855 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:48.881855 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.881855 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:48.882904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.882904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:48.882904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:48.882904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:48.882904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.882904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:29:48.882904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:48.882904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:48.882904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:48.882904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:48.882904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:48.882904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:48.883267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:48.883267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:48.883267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:48.883267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:48.883267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:48.883267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:48.883267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:48.883267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:48.883267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:48.883267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:48.883267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:48.883267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:48.883267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:48.883729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.883729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:48.884120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.884120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:48.884120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:48.884120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:48.884120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:48.884120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:48.884120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.884120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:48.884120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:48.884120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.884120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:48.884120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:48.884120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.884120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:48.884120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:48.884120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.885130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:29:48.885130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:48.885130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:48.885130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:48.885130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:48.885130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:48.885130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:48.885130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:48.885130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:48.885130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:48.885130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:48.885130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:48.885130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:48.885130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:48.885130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:48.885130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:48.885130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:48.885130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:48.885130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:48.887332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:48.887332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.887332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:48.929534 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:48.930091 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:48.930280 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:48.930456 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:29:48.931233 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:48.931586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.932435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:48.932435 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:48.932435 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:48.932435 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:48.932435 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:48.932435 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:48.932435 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.932435 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:48.933717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.933717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:29:48.933717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:48.933717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:48.933717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:48.933717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:48.933717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:48.933717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:48.933717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:48.933717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:48.933717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:48.933717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:29:48.933717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:48.933717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:29:48.933717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:29:48.934120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:48.934120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:29:48.934120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:48.934120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:29:48.934120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:29:48.934120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:48.934120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:29:48.934120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.934120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:29:48.935141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.935141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:48.935141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:48.935141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:48.935141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.935141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:29:48.935141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:29:48.935141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:29:48.935141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:48.935141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:48.935141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:48.935141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:48.935479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:48.935479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:48.935479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:48.935479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:48.935479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:48.935479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:48.935479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:48.935479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:48.935479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:48.935479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:48.935479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:48.935479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:48.935479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:48.935479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:48.935947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:48.935947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:48.935947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.935947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:48.936683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.936683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:48.936683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:48.936683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:48.936683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:48.936683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:48.936683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.936683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:48.936683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:48.936683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.936683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:48.936683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:48.936683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.936683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:48.936683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:48.936683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.937018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:29:48.937018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:29:48.937018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:29:48.937018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:48.937018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:48.937018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:48.937018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:48.937018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:48.937018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:48.937018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:48.937018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:48.937018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:48.937018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:48.937018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:48.937018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:48.937018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:48.937018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:48.937018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:48.937337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:48.937337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:48.937337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:48.937337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:48.937337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:48.937337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.937337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:48.939777 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:48.940061 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:48.940167 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:48.940300 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:48.940874 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:48.941371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.942312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:48.942312 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:48.942312 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:48.942312 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:48.942312 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:48.942312 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:48.942312 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.942312 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:48.943144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.943144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:29:48.943144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:48.943144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:48.943144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:48.943144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:48.943144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:48.943144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:48.943144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:48.943144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:48.943144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:48.943144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:29:48.943144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:48.943144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:29:48.943144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:29:48.943421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:48.943421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:29:48.943421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:48.943421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:29:48.943421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:29:48.943421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:48.943421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:29:48.943421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.943421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:29:48.944046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.944046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:48.944046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:48.944046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:48.944046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.944046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:29:48.944046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:29:48.944046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:48.944046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:48.944046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:48.944046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:48.944357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:48.944357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:48.944357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:48.944357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:48.944357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:48.944357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:48.944357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:48.944357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:48.944357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:48.944357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:48.944357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:48.944357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:48.944357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:48.944357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:48.944703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:48.944703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:48.944703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.944703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:48.945300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.945300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:48.945300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:48.945300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:48.945300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:48.945300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:48.945300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.945300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:48.945300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:48.945300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.945300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:48.945300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:48.945300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.945300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:48.945300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:48.945300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.945300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:29:48.946175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:29:48.946175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:48.946175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:48.946175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:48.946175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:48.946175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:48.946175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:48.946175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:48.946175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:48.946175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:48.946175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:48.946175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:48.946175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:48.946175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:48.946175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:48.946175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:48.946175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:48.946175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:48.946175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:48.946637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:48.946637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:48.946637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.946637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:48.951166 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:48.951417 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:48.951532 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:48.951912 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:48.952224 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:48.952509 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:48.952705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.953261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:48.953261 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:48.953261 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:48.953261 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:48.953261 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:48.953261 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:48.953261 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.953261 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:48.954098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.954098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:48.954098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:48.954098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:48.954098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.954098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:29:48.954098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:48.954098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:48.954098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:48.954098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:48.954098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:48.954098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:48.954748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:48.954748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:48.954748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:48.954748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:48.954748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:48.954748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:48.954748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:48.954748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:48.954748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:48.954748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:48.954748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:48.954748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:48.954748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:48.955407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.955407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:48.955407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.955407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:48.955407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:48.955407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:48.955407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:48.955407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:48.955407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.955407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:48.955407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:48.955407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.955407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:48.955407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:48.955407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.956163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:48.956163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:48.956163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:48.956163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:29:48.956163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:48.956163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:48.956163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:48.956163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:48.956163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:48.956163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:48.956163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:48.956163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:48.956163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:48.956163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:48.956163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:48.956163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:48.956163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:48.956163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:48.956163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:48.957067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:48.957067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:48.957067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:48.957067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:48.957067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:48.957067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:53.642433 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:53.643417 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:53.643417 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:53.644767 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:53.679605 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:53.679884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.680556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:53.680556 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:53.680556 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:53.680556 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:53.680556 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:53.680556 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:53.680556 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.680556 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:53.681634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.681634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:53.681634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:53.681634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:53.681634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.681634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:29:53.681634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:53.681634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:53.681634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:53.681634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:53.681634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:53.681634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:53.682360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:53.682360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:53.682360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:53.682360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:53.682360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:53.682360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:53.682360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:53.682360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:53.682360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:53.682360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:53.682360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:53.682360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:53.682360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:53.683063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.683063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:53.683063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.683063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:53.683063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:53.683063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:53.683063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:53.683063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:53.683063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.683063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:53.683063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:53.683063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.683063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:53.683063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:53.683063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.683901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:53.683901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:53.683901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.683901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:29:53.683901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:53.683901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:53.683901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:53.683901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:53.683901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:53.683901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:53.683901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:53.683901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:53.683901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:53.683901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:53.683901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:53.683901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:53.683901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:53.683901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:53.683901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:53.685072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:53.685072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:53.685072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:53.685072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:53.685072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.685072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:53.877624 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:53.878258 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:53.878555 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:53.879741 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:53.880013 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:53.880184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.880978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:53.880978 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:53.880978 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:53.880978 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:53.880978 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:53.880978 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:53.880978 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.880978 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:53.882400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.882400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:53.882400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:53.882400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:53.882400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.882400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:29:53.882400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:53.882400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:53.882400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:53.882400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:53.882400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:53.882400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:53.883013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:53.883013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:53.883013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:53.883013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:53.883013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:53.883013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:53.883013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:53.883013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:53.883013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:53.883013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:53.883013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:53.883013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:53.883013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:53.883595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.883595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:53.884063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.884063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:53.884063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:53.884063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:53.884063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:53.884063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:53.884063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.884063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:53.884063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:53.884063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.884063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:53.884063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:53.884063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.884063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:53.884063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:53.884063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.884820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:29:53.884820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:53.884820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:53.884820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:53.884820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:53.884820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:53.884820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:53.884820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:53.884820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:53.884820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:53.884820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:53.884820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:53.884820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:53.884820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:53.884820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:53.884820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:53.884820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:53.884820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:53.884820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:53.885730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:53.885730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.885730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:53.930207 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:53.931137 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:53.931377 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:53.931638 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:29:53.932369 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:53.932603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.933295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:53.933295 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:53.933295 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:53.933295 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:53.933295 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:53.933295 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:53.933295 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.933295 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:53.934363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.934363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:29:53.934363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:53.934363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:53.934363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:53.934363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:53.934363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:53.934363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:53.934363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:53.934363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:53.934363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:53.934363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:29:53.934363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:53.934363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:29:53.934363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:29:53.934897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:53.934897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:29:53.934897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:53.934897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:29:53.934897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:29:53.934897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:53.934897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:29:53.934897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.934897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:29:53.935320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.935320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:53.935320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:53.935320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:53.935320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.935320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:29:53.935320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:29:53.935320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:29:53.935320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:53.935320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:53.935320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:53.935320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:53.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:53.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:53.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:53.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:53.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:53.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:53.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:53.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:53.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:53.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:53.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:53.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:53.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:53.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:53.936120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:53.936120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:53.936120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.936120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:53.936482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.936482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:53.936482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:53.936482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:53.936482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:53.936482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:53.936482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.936482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:53.936482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:53.936482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.936482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:53.936482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:53.936482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.936482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:53.936482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:53.936482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.936961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:29:53.936961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:29:53.936961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:29:53.936961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:53.936961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:53.936961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:53.936961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:53.936961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:53.936961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:53.936961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:53.936961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:53.936961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:53.936961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:53.936961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:53.936961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:53.936961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:53.936961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:53.936961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:53.937502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:53.937502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:53.937502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:53.937502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:53.937502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:53.937502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.937502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:53.940285 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:53.940570 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:53.940694 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:53.940865 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:53.941234 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:53.941510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.942001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:53.942001 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:53.942001 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:53.942001 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:53.942001 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:53.942001 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:53.942001 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.942001 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:53.942799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.942799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:29:53.942799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:53.942799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:53.942799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:53.942799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:53.942799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:53.942799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:53.942799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:53.942799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:53.942799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:53.942799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:29:53.942799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:53.942799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:29:53.942799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:29:53.943275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:53.943275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:29:53.943275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:53.943275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:29:53.943275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:29:53.943275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:53.943275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:29:53.943275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.943275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:29:53.943690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.943690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:53.943690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:53.943690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:53.943690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.943690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:29:53.943690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:29:53.943690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:53.943690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:53.943690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:53.943690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:53.944026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:53.944026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:53.944026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:53.944026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:53.944026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:53.944026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:53.944026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:53.944026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:53.944026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:53.944026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:53.944026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:53.944026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:53.944026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:53.944026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:53.944418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:53.944418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:53.944418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.944418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:53.944873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.944873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:53.944873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:53.944873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:53.944873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:53.944873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:53.944873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.944873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:53.944873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:53.944873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.944873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:53.944873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:53.944873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.944873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:53.944873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:53.944873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.944873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:29:53.945344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:29:53.945344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:53.945344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:53.945344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:53.945344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:53.945344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:53.945344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:53.945344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:53.945344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:53.945344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:53.945344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:53.945344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:53.945344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:53.945344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:53.945344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:53.945344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:53.945344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:53.945344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:53.945344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:53.945894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:53.945894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:53.945894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.945894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:53.951523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:53.951818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:53.951935 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:53.952115 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:53.953246 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:53.953518 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:53.953644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.954416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:53.954416 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:53.954416 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:53.954416 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:53.954416 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:53.954416 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:53.954416 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.954416 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:53.955402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.955402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:53.955402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:53.955402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:53.955402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.955402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:29:53.955402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:53.955402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:53.955402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:53.955402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:53.955402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:53.955402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:53.955769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:53.955769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:53.955769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:53.955769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:53.955769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:53.955769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:53.955769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:53.955769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:53.955769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:53.955769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:53.955769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:53.955769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:53.955769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:53.956225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.956225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:53.956395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.956395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:53.956395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:53.956395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:53.956395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:53.956395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:53.956395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.956395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:53.956395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:53.956395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.956395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:53.956395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:53.956395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.956395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:53.956395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:53.956395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:53.956846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:29:53.956846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:53.956846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:53.956846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:53.956846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:53.956846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:53.956846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:53.956846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:53.956846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:53.956846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:53.956846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:53.956846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:53.956846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:53.956846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:53.956846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:53.956846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:53.956846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:53.956846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:53.956846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:53.957356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:53.957356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:53.957356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:58.643609 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:58.644467 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:58.644720 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:58.646064 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:58.680938 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:58.681394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.682543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:58.682543 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:58.682543 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:58.682543 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:58.682543 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:58.682543 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:58.682543 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.682543 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:58.683401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.683401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:58.683401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:58.683401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:58.683401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.683401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:29:58.683401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:58.683401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:58.683401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:58.683401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:58.683401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:58.683401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:58.683773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:58.683773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:58.683773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:58.683773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:58.683773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:58.683773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:58.683773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:58.683773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:58.683773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:58.683773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:58.683773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:58.683773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:58.683773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:58.684354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.684354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:58.684592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.684592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:58.684592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:58.684592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:58.684592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:58.684592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:58.684592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.684592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:58.684592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:58.684592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.684592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:58.684592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:58.684592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.684592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:58.684592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:58.684592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.685055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:29:58.685055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:58.685055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:58.685055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:58.685055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:58.685055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:58.685055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:58.685055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:58.685055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:58.685055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:58.685055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:58.685055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:58.685055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:58.685055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:58.685055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:58.685055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:58.685055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:58.685055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:58.685055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:58.685564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:58.685564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.685564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:58.878346 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:58.879018 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:58.879262 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:58.880698 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:58.881063 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:58.881305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.882302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:58.882302 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:58.882302 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:58.882302 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:58.882302 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:58.882302 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:58.882302 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.882302 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:58.884445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.884445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:58.884445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:58.884445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:58.884445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.884445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:29:58.884445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:58.884445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:58.884445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:58.884445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:58.884445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:58.884445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:58.885077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:58.885077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:58.885077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:58.885077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:58.885077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:58.885077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:58.885077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:58.885077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:58.885077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:58.885077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:58.885077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:58.885077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:58.885077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:58.885638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.885638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:58.886421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.886421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:58.886421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:58.886421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:58.886421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:58.886421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:58.886421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.886421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:58.886421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:58.886421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.886421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:58.886421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:58.886421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.886421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:58.886421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:58.886421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:29:58.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:58.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:58.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:58.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:58.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:58.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:58.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:58.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:58.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:58.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:58.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:58.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:58.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:58.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:58.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:58.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:58.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:58.887264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:58.888166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:58.888166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.888166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:58.931158 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:58.931666 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:58.931934 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:58.932189 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:29:58.933078 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:58.933432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.934437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:58.934437 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:58.934437 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:58.934437 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:58.934437 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:58.934437 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:58.934437 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.934437 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:58.935786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.935786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:29:58.935786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:58.935786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:58.935786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:58.935786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:58.935786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:58.935786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:58.935786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:58.935786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:58.935786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:58.935786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:29:58.935786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:58.935786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:29:58.935786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:29:58.936648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:58.936648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:29:58.936648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:58.936648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:29:58.936648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:29:58.936648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:58.936648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:29:58.936648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.936648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:29:58.937154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.937154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:58.937154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:58.937154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:58.937154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.937154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:29:58.937154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:29:58.937154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:29:58.937154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:58.937154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:58.937154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:58.937154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:58.937750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:58.937750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:58.937750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:58.937750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:58.937750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:58.937750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:58.937750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:58.937750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:58.937750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:58.937750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:58.937750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:58.937750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:58.937750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:58.937750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:58.938421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:58.938421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:58.938421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.938421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:58.938980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.938980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:58.938980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:58.938980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:58.938980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:58.938980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:58.938980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.938980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:58.938980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:58.938980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.938980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:58.938980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:58.938980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.938980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:58.938980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:58.938980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.939975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:29:58.939975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:29:58.939975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:29:58.939975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:58.939975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:58.939975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:58.939975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:58.939975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:58.939975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:58.939975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:58.939975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:58.939975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:58.939975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:58.939975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:58.939975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:58.939975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:58.939975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:58.939975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:58.940855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:58.940855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:58.940855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:58.940855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:58.940855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:58.940855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.940855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:58.940855 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:58.941237 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:58.941237 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:58.941237 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:58.941924 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:58.942429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.943176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:58.943176 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:58.943176 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:58.943176 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:58.943176 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:58.943176 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:58.943176 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.943176 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:58.943882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.943882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:29:58.943882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:58.943882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:58.943882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:58.943882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:58.943882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:58.943882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:58.943882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:58.943882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:29:58.943882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:58.943882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:29:58.943882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:58.943882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:29:58.943882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:29:58.944355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:58.944355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:29:58.944355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:58.944355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:29:58.944355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:29:58.944355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:58.944355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:29:58.944355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.944355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:29:58.944744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.944744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:58.944744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:58.944744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:58.944744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.944744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:29:58.944744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:29:58.944744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:58.944744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:58.944744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:58.944744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:58.945123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:58.945123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:58.945123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:58.945123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:58.945123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:58.945123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:58.945123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:58.945123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:58.945123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:58.945123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:58.945123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:58.945123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:58.945123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:58.945123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:58.945530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:58.945530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:58.945530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.945530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:58.945800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.945800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:58.945800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:58.945800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:58.945800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:58.945800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:58.945800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.945800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:58.945800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:58.945800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.945800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:58.945800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:58.945800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.945800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:58.945800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:58.945800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.945800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:29:58.946337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:29:58.946337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:29:58.946337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:29:58.946337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:58.946337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:58.946337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:58.946337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:58.946337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:58.946337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:58.946337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:58.946337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:58.946337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:58.946337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:58.946337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:58.946337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:58.946337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:58.946337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:58.946337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:58.946337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:58.946914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:58.946914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:58.946914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.946914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:29:58.952035 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:29:58.952347 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:29:58.952460 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:29:58.952608 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:29:58.952925 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:58.953144 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:29:58.953293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.953811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:29:58.953811 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:58.953811 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:29:58.953811 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:29:58.953811 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:58.953811 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:29:58.953811 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.953811 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:29:58.954643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.954643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:29:58.954643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:58.954643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:29:58.954643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.954643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:29:58.954643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:58.954643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:58.954643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:58.954643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:58.954643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:58.954643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:58.955055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:29:58.955055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:58.955055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:29:58.955055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:58.955055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:29:58.955055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:29:58.955055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:58.955055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:29:58.955055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:58.955055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:29:58.955055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:29:58.955055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:58.955055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:29:58.955388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.955388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:29:58.955670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.955670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:29:58.955670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:29:58.955670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:29:58.955670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:29:58.955670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:29:58.955670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.955670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:29:58.955670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:29:58.955670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.955670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:29:58.955670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:29:58.955670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.955670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:29:58.955670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:29:58.955670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:29:58.956140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:29:58.956140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:29:58.956140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:58.956140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:58.956140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:29:58.956140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:29:58.956140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:29:58.956140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:29:58.956140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:29:58.956140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:29:58.956140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:29:58.956140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:29:58.956140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:29:58.956140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:29:58.956140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:29:58.956140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:29:58.956140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:29:58.956140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:29:58.956140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:29:58.956653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:29:58.956653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:29:58.956653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:03.644373 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:03.645139 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:03.645139 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:03.646271 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:03.647975 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=239 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:30:03.677619 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:03.678125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.679059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:03.679059 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:03.679059 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:03.679059 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:03.679059 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:03.679059 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:03.679059 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.679059 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:03.679810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.679810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:03.679810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:03.679810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:03.679810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.679810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:30:03.679810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:03.679810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:03.679810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:03.679810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:03.679810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:03.679810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:03.680433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:03.680433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:03.680433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:03.680433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:03.680433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:03.680433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:03.680433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:03.680433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:03.680433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:03.680433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:03.680433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:03.680433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:03.680433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:03.681023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.681023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:03.681023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.681023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:03.681023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:03.681023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:03.681023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:03.681023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:03.681023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.681023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:03.681023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:03.681023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.681023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:03.681023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:03.681023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.682055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:03.682055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:03.682055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.682055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:30:03.682055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:03.682055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:03.682055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:03.682055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:03.682055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:03.682055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:03.682055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:03.682055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:03.682055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:03.682055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:03.682055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:03.682055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:03.682055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:03.682055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:03.682055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:03.683347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:03.683347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:03.683347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:03.683347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:03.683347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.683347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:03.879036 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:03.879379 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:03.879589 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:03.880653 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:03.880909 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:03.881048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.881706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:03.881706 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:03.881706 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:03.881706 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:03.881706 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:03.881706 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:03.881706 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.881706 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:03.883271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.883271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:03.883271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:03.883271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:03.883271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.883271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:30:03.883271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:03.883271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:03.883271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:03.883271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:03.883271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:03.883271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:03.883782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:03.883782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:03.883782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:03.883782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:03.883782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:03.883782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:03.883782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:03.883782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:03.883782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:03.883782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:03.883782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:03.883782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:03.883782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:03.884245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.884245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:03.885072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.885072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:03.885072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:03.885072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:03.885072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:03.885072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:03.885072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.885072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:03.885072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:03.885072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.885072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:03.885072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:03.885072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.885072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:03.885072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:03.885072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.885664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:30:03.885664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:03.885664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:03.885664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:03.885664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:03.885664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:03.885664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:03.885664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:03.885664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:03.885664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:03.885664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:03.885664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:03.885664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:03.885664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:03.885664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:03.885664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:03.885664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:03.885664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:03.885664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:03.886375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:03.886375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.886375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:03.931798 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:03.932238 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:03.932476 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:03.932709 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:30:03.933331 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:03.933747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.935434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:03.935434 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:03.935434 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:03.935434 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:03.935434 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:03.935434 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:03.935434 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.935434 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:03.936872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.936872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:30:03.936872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:03.936872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:03.936872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:03.936872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:03.936872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:03.936872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:03.936872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:03.936872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:03.936872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:03.936872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:30:03.936872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:03.936872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:30:03.936872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:30:03.937509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:03.937509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:30:03.937509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:03.937509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:30:03.937509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:30:03.937509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:03.937509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:30:03.937509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.937509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:30:03.938287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.938287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:03.938287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:03.938287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:03.938287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.938287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:30:03.938287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:30:03.938287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:30:03.938287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:03.938287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:03.938287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:03.938287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:03.938673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:03.938673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:03.938673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:03.938673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:03.938673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:03.938673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:03.938673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:03.938673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:03.938673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:03.938673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:03.938673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:03.938673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:03.938673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:03.938673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:03.939065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:03.939065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:03.939065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.939065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:03.940039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.940039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:03.940039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:03.940039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:03.940039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:03.940039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:03.940039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.940039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:03.940039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:03.940039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.940039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:03.940039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:03.940039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.940039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:03.940039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:03.940039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.940541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:30:03.940541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:30:03.940541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:30:03.940541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:03.940541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:03.940541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:03.940541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:03.940541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:03.940541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:03.940541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:03.940541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:03.940541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:03.940541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:03.940541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:03.940541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:03.940541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:03.940541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:03.940541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:03.941100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:03.941100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:03.941100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:03.941100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:03.941100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:03.941100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.941100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:03.941313 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:03.941494 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:03.941641 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:03.941799 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:03.942168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:03.942419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.942960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:03.942960 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:03.942960 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:03.942960 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:03.942960 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:03.942960 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:03.942960 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.942960 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:03.943741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.943741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:30:03.943741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:03.943741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:03.943741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:03.943741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:03.943741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:03.943741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:03.943741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:03.943741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:03.943741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:03.943741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:30:03.943741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:03.943741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:30:03.943741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:30:03.944247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:03.944247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:30:03.944247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:03.944247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:30:03.944247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:30:03.944247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:03.944247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:30:03.944247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.944247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:30:03.944693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.944693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:03.944693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:03.944693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:03.944693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.944693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:30:03.944693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:30:03.944693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:03.944693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:03.944693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:03.944693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:03.945033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:03.945033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:03.945033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:03.945033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:03.945033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:03.945033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:03.945033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:03.945033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:03.945033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:03.945033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:03.945033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:03.945033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:03.945033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:03.945033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:03.945505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:03.945505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:03.945505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.945505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:03.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:03.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:03.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:03.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:03.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:03.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:03.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:03.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:03.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:03.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:03.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:03.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.945828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:30:03.946326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:30:03.946326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:03.946326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:03.946326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:03.946326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:03.946326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:03.946326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:03.946326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:03.946326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:03.946326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:03.946326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:03.946326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:03.946326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:03.946326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:03.946326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:03.946326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:03.946326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:03.946326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:03.946326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:03.946846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:03.946846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:03.946846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.946846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:03.952615 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:03.953077 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:03.953197 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:03.953351 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:03.953897 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:03.954297 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:03.954578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.955356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:03.955356 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:03.955356 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:03.955356 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:03.955356 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:03.955356 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:03.955356 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.955356 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:03.956201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.956201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:03.956201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:03.956201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:03.956201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.956201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:30:03.956201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:03.956201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:03.956201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:03.956201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:03.956201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:03.956201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:03.956923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:03.956923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:03.956923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:03.956923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:03.956923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:03.956923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:03.956923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:03.956923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:03.956923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:03.956923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:03.956923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:03.956923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:03.956923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:03.957474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.957474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:03.957474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.957474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:03.957474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:03.957474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:03.957474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:03.957474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:03.957474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.957474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:03.957474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:03.957474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.957474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:03.957474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:03.957474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.958227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:03.958227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:03.958227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:03.958227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:30:03.958227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:03.958227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:03.958227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:03.958227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:03.958227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:03.958227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:03.958227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:03.958227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:03.958227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:03.958227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:03.958227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:03.958227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:03.958227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:03.958227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:03.958227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:03.959041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:03.959041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:03.959041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:03.959041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:03.959041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:03.959041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:08.645111 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:08.646265 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:08.646450 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:08.647640 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:08.688107 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:08.688107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.688329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:08.688329 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:08.688329 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:08.688329 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:08.688329 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:08.688329 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:08.688329 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.688329 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:08.689353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.689353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:08.689353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:08.689353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:08.689353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.689353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:30:08.689353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:08.689353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:08.689353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:08.689353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:08.689353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:08.689353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:08.690251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:08.690251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:08.690251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:08.690251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:08.690251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:08.690251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:08.690251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:08.690251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:08.690251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:08.690251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:08.690251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:08.690251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:08.690251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:08.691323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.691323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:08.691323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.691323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:08.691323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:08.691323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:08.691323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:08.691323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:08.691323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.691323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:08.691323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:08.691323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.691323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:08.691323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:08.691323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.692109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:08.692109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:08.692109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.692109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:30:08.692109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:08.692109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:08.692109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:08.692109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:08.692109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:08.692109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:08.692109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:08.692109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:08.692109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:08.692109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:08.692109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:08.692109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:08.692109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:08.692109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:08.692109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:08.693989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:08.693989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:08.693989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:08.693989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:08.693989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.693989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:08.879670 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:08.880297 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:08.880489 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:08.882759 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:08.882759 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:08.882759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.883747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:08.883747 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:08.883747 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:08.883747 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:08.883747 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:08.883747 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:08.883747 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.883747 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:08.884649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.884649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:08.884649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:08.884649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:08.884649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.884649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:30:08.884649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:08.884649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:08.884649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:08.884649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:08.884649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:08.884649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:08.885382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:08.885382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:08.885382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:08.885382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:08.885382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:08.885382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:08.885382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:08.885382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:08.885382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:08.885382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:08.885382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:08.885382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:08.885382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:08.886149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.886149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:08.886371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.886371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:08.886371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:08.886371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:08.886371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:08.886371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:08.886371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.886371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:08.886371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:08.886371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.886371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:08.886371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:08.886371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.886371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:08.886371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:08.886371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.887623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:30:08.887623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:08.887623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:08.887623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:08.887623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:08.887623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:08.887623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:08.887623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:08.887623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:08.887623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:08.887623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:08.887623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:08.887623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:08.887623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:08.887623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:08.887623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:08.887623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:08.887623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:08.887623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:08.888190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:08.888190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.888190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:08.932384 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:08.932943 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:08.933083 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:08.933212 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:30:08.934846 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:08.934846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.935317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:08.935317 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:08.935317 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:08.935317 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:08.935317 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:08.935317 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:08.935317 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.935317 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:08.936115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.936115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:30:08.936115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:08.936115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:08.936115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:08.936115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:08.936115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:08.936115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:08.936115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:08.936115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:08.936115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:08.936115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:30:08.936115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:08.936115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:30:08.936115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:30:08.936811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:08.936811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:30:08.936811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:08.936811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:30:08.936811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:30:08.936811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:08.936811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:30:08.936811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.936811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:30:08.937184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.937184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:08.937184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:08.937184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:08.937184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.937184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:30:08.937184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:30:08.937184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:30:08.937184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:08.937184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:08.937184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:08.937184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:08.937660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:08.937660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:08.937660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:08.937660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:08.937660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:08.937660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:08.937660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:08.937660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:08.937660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:08.937660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:08.937660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:08.937660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:08.937660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:08.937660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:08.938281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:08.938281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:08.938281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.938281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:08.938281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.938281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:08.938281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:08.938281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:08.938281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:08.938281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:08.938281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.938281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:08.938281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:08.938281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.938848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:08.938848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:08.938848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.938848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:08.938848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:08.938848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.938848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:30:08.938848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:30:08.938848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:30:08.938848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:08.938848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:08.938848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:08.938848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:08.938848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:08.938848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:08.938848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:08.938848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:08.938848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:08.938848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:08.939596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:08.939596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:08.939596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:08.939596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:08.939596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:08.939596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:08.939596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:08.939596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:08.939596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:08.939596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:08.939596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.939596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:08.941720 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:08.942147 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:08.942405 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:08.943496 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:08.943496 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:08.943728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.944410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:08.944410 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:08.944410 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:08.944410 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:08.944410 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:08.944410 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:08.944410 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.944410 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:08.945231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.945231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:30:08.945231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:08.945231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:08.945231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:08.945231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:08.945231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:08.945231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:08.945231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:08.945231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:08.945231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:08.945231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:30:08.945231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:08.945231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:30:08.945231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:30:08.946085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:08.946085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:30:08.946085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:08.946085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:30:08.946085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:30:08.946085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:08.946085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:30:08.946085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.946085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:30:08.946551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.946551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:08.946551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:08.946551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:08.946551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.946551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:30:08.946551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:30:08.946551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:08.946551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:08.946551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:08.946551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:08.947215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:08.947215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:08.947215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:08.947215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:08.947215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:08.947215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:08.947215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:08.947215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:08.947215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:08.947215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:08.947215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:08.947215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:08.947215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:08.947215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:08.947997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:08.947997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:08.947997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.947997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:08.947997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.947997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:08.947997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:08.947997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:08.947997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:08.947997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:08.947997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.947997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:08.947997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:08.947997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.948784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:08.948784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:08.948784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.948784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:08.948784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:08.948784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.948784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:30:08.948784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:30:08.948784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:08.948784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:08.948784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:08.948784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:08.948784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:08.948784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:08.948784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:08.948784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:08.948784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:08.948784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:08.948784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:08.949811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:08.949811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:08.949811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:08.949811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:08.949811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:08.949811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:08.949811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:08.949811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:08.949811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:08.949811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.949811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:08.953279 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:08.953574 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:08.953699 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:08.954827 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:08.954827 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:08.954827 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:08.955201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.955974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:08.955974 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:08.955974 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:08.955974 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:08.955974 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:08.955974 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:08.955974 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.955974 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:08.956784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.956784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:08.956784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:08.956784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:08.956784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.956784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:30:08.956784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:08.956784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:08.956784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:08.956784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:08.956784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:08.956784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:08.957420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:08.957420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:08.957420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:08.957420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:08.957420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:08.957420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:08.957420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:08.957420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:08.957420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:08.957420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:08.957420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:08.957420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:08.957420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:08.958224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.958224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:08.958224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.958224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:08.958224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:08.958224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:08.958224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:08.958224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:08.958224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.958224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:08.958224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:08.958224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.958224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:08.958224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:08.958224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.960450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:08.960450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:08.960450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:08.960450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:30:08.960450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:08.960450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:08.960450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:08.960450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:08.960450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:08.960450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:08.960450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:08.960450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:08.960450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:08.960450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:08.960450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:08.960450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:08.960450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:08.960450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:08.960450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:08.962258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:08.962258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:08.962258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:08.962258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:08.962258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:08.962258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:13.646533 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:13.647168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:13.647241 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:13.648523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:13.681146 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:13.681695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.682695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:13.682695 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:13.682695 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:13.682695 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:13.682695 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:13.682695 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:13.682695 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.682695 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:13.683770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.683770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:13.683770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:13.683770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:13.683770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.683770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:30:13.683770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:13.683770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:13.683770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:13.683770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:13.683770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:13.683770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:13.684404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:13.684404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:13.684404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:13.684404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:13.684404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:13.684404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:13.684404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:13.684404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:13.684404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:13.684404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:13.684404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:13.684404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:13.684404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:13.684759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.684759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:13.684759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.684759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:13.684759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:13.684759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:13.684759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:13.684759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:13.684759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.684759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:13.684759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:13.684759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.684759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:13.684759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:13.684759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.685146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:13.685146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:13.685146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.685146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:30:13.685146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:13.685146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:13.685146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:13.685146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:13.685146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:13.685146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:13.685146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:13.685146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:13.685146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:13.685146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:13.685146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:13.685146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:13.685146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:13.685146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:13.685146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:13.685653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:13.685653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:13.685653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:13.685653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:13.685653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.685653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:13.880247 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:13.880888 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:13.881104 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:13.882213 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:13.882403 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:13.882634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.883519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:13.883519 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:13.883519 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:13.883519 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:13.883519 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:13.883519 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:13.883519 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.883519 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:13.885090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.885090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:13.885090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:13.885090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:13.885090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.885090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:30:13.885090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:13.885090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:13.885090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:13.885090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:13.885090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:13.885090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:13.885378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:13.885378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:13.885378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:13.885378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:13.885378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:13.885378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:13.885378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:13.885378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:13.885378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:13.885378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:13.885378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:13.885378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:13.885378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:13.885645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.885645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:13.887948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.887948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:13.887948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:13.887948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:13.887948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:13.887948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:13.887948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.887948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:13.887948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:13.887948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.887948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:13.887948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:13.887948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.887948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:13.887948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:13.887948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.888395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:30:13.888395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:13.888395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:13.888395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:13.888395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:13.888395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:13.888395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:13.888395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:13.888395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:13.888395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:13.888395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:13.888395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:13.888395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:13.888395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:13.888395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:13.888395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:13.888395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:13.888395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:13.888395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:13.889028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:13.889028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.889028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:13.933236 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:13.933770 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:13.934458 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:13.934458 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:30:13.934848 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:13.935087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.936125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:13.936125 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:13.936125 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:13.936125 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:13.936125 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:13.936125 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:13.936125 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.936125 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:13.937519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.937519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:30:13.937519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:13.937519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:13.937519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:13.937519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:13.937519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:13.937519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:13.937519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:13.937519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:13.937519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:13.937519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:30:13.937519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:13.937519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:30:13.937519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:30:13.938262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:13.938262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:30:13.938262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:13.938262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:30:13.938262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:30:13.938262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:13.938262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:30:13.938262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.938262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:30:13.938882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.938882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:13.938882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:13.938882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:13.938882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.938882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:30:13.938882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:30:13.938882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:30:13.938882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:13.938882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:13.938882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:13.938882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:13.939468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:13.939468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:13.939468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:13.939468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:13.939468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:13.939468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:13.939468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:13.939468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:13.939468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:13.939468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:13.939468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:13.939468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:13.939468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:13.939468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:13.940050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:13.940050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:13.940050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.940050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:13.940418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.940418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:13.940418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:13.940418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:13.940418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:13.940418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:13.940418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.940418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:13.940418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:13.940418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.940418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:13.940418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:13.940418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.940418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:13.940418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:13.940418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.941088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:30:13.941088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:30:13.941088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:30:13.941088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:13.941088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:13.941088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:13.941088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:13.941088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:13.941088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:13.941088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:13.941088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:13.941088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:13.941088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:13.941088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:13.941088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:13.941088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:13.941088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:13.941088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:13.942155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:13.942155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:13.942155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:13.942155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:13.942155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:13.942155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.942155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:13.942452 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:13.942527 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:13.942641 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:13.942796 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:13.943238 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:13.943557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.944261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:13.944261 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:13.944261 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:13.944261 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:13.944261 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:13.944261 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:13.944261 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.944261 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:13.945117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.945117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:30:13.945117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:13.945117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:13.945117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:13.945117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:13.945117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:13.945117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:13.945117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:13.945117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:13.945117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:13.945117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:30:13.945117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:13.945117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:30:13.945117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:30:13.945405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:13.945405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:30:13.945405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:13.945405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:30:13.945405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:30:13.945405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:13.945405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:30:13.945405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.945405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:30:13.945949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.945949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:13.945949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:13.945949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:13.945949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.945949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:30:13.945949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:30:13.945949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:13.945949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:13.945949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:13.945949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:13.946439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:13.946439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:13.946439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:13.946439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:13.946439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:13.946439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:13.946439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:13.946439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:13.946439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:13.946439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:13.946439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:13.946439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:13.946439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:13.946439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:13.946703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:13.946703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:13.946703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.946703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:13.947024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.947024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:13.947024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:13.947024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:13.947024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:13.947024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:13.947024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.947024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:13.947024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:13.947024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.947024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:13.947024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:13.947024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.947024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:13.947024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:13.947024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.947024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:30:13.947860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:30:13.947860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:13.947860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:13.947860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:13.947860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:13.947860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:13.947860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:13.947860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:13.947860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:13.947860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:13.947860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:13.947860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:13.947860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:13.947860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:13.947860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:13.947860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:13.947860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:13.947860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:13.947860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:13.948277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:13.948277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:13.948277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.948277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:13.953959 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:13.954236 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:13.954348 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:13.954537 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:13.954956 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:13.955288 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:13.955504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.956336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:13.956336 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:13.956336 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:13.956336 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:13.956336 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:13.956336 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:13.956336 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.956336 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:13.957513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.957513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:13.957513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:13.957513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:13.957513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.957513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:30:13.957513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:13.957513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:13.957513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:13.957513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:13.957513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:13.957513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:13.957898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:13.957898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:13.957898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:13.957898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:13.957898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:13.957898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:13.957898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:13.957898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:13.957898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:13.957898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:13.957898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:13.957898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:13.957898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:13.958237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.958237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:13.959208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.959208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:13.959208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:13.959208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:13.959208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:13.959208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:13.959208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.959208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:13.959208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:13.959208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.959208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:13.959208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:13.959208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.959208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:13.959208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:13.959208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:13.959593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:30:13.959593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:13.959593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:13.959593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:13.959593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:13.959593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:13.959593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:13.959593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:13.959593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:13.959593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:13.959593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:13.959593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:13.959593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:13.959593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:13.959593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:13.959593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:13.959593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:13.959593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:13.959593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:13.960080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:13.960080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:13.960080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:18.647342 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:18.648206 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:18.648206 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:18.649481 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:18.686301 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:18.686477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.687134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:18.687134 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:18.687134 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:18.687134 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:18.687134 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:18.687134 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:18.687134 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.687134 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:18.687904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.687904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:18.687904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:18.687904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:18.687904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.687904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:30:18.687904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:18.687904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:18.687904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:18.687904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:18.687904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:18.687904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:18.688555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:18.688555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:18.688555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:18.688555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:18.688555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:18.688555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:18.688555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:18.688555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:18.688555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:18.688555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:18.688555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:18.688555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:18.688555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:18.689225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.689225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:18.689225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.689225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:18.689225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:18.689225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:18.689225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:18.689225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:18.689225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.689225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:18.689225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:18.689225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.689225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:18.689225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:18.689225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.689977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:18.689977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:18.689977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.689977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:30:18.689977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:18.689977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:18.689977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:18.689977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:18.689977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:18.689977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:18.689977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:18.689977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:18.689977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:18.689977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:18.689977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:18.689977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:18.689977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:18.689977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:18.689977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:18.690889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:18.690889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:18.690889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:18.690889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:18.690889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.690889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:18.881026 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:18.881276 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:18.881276 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:18.882398 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:18.882885 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:18.883222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.884166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:18.884166 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:18.884166 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:18.884166 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:18.884166 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:18.884166 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:18.884166 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.884166 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:18.885157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.885157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:18.885157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:18.885157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:18.885157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.885157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:30:18.885157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:18.885157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:18.885157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:18.885157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:18.885157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:18.885157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:18.885545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:18.885545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:18.885545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:18.885545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:18.885545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:18.885545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:18.885545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:18.885545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:18.885545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:18.885545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:18.885545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:18.885545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:18.885545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:18.885924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.885924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:18.886323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.886323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:18.886323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:18.886323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:18.886323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:18.886323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:18.886323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.886323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:18.886323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:18.886323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.886323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:18.886323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:18.886323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.886323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:18.886323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:18.886323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.887210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:30:18.887210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:18.887210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:18.887210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:18.887210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:18.887210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:18.887210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:18.887210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:18.887210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:18.887210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:18.887210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:18.887210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:18.887210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:18.887210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:18.887210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:18.887210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:18.887210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:18.887210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:18.887210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:18.887962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:18.887962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.887962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:18.933752 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:18.934054 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:18.934182 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:18.934320 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:30:18.935061 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:18.935357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.936079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:18.936079 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:18.936079 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:18.936079 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:18.936079 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:18.936079 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:18.936079 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.936079 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:18.937072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.937072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:30:18.937072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:18.937072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:18.937072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:18.937072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:18.937072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:18.937072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:18.937072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:18.937072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:18.937072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:18.937072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:30:18.937072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:18.937072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:30:18.937072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:30:18.937668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:18.937668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:30:18.937668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:18.937668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:30:18.937668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:30:18.937668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:18.937668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:30:18.937668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.937668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:30:18.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:18.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:18.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:18.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:30:18.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:30:18.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:30:18.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:18.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:18.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:18.937985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:18.938371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:18.938371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:18.938371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:18.938371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:18.938371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:18.938371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:18.938371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:18.938371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:18.938371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:18.938371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:18.938371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:18.938371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:18.938371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:18.938371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:18.938736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:18.938736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:18.938736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.938736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:18.938832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.938832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:18.938832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:18.938832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:18.938832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:18.938832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:18.938832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.938832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:18.938832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:18.938832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.938832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:18.938832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:18.938832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.938832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:18.938832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:18.938832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.939149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:30:18.939149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:30:18.939149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:30:18.939149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:18.939149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:18.939149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:18.939149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:18.939149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:18.939149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:18.939149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:18.939149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:18.939149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:18.939149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:18.939149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:18.939149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:18.939149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:18.939149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:18.939149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:18.939502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:18.939502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:18.939502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:18.939502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:18.939502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:18.939502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.939502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:18.942644 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:18.942927 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:18.943053 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:18.943179 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:18.943643 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:18.943996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.944634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:18.944634 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:18.944634 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:18.944634 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:18.944634 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:18.944634 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:18.944634 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.944634 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:18.945370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.945370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:30:18.945370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:18.945370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:18.945370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:18.945370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:18.945370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:18.945370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:18.945370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:18.945370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:18.945370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:18.945370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:30:18.945370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:18.945370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:30:18.945370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:30:18.945707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:18.945707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:30:18.945707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:18.945707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:30:18.945707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:30:18.945707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:18.945707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:30:18.945707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.945707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:30:18.946238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.946238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:18.946238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:18.946238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:18.946238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.946238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:30:18.946238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:30:18.946238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:18.946238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:18.946238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:18.946238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:18.946487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:18.946487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:18.946487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:18.946487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:18.946487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:18.946487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:18.946487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:18.946487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:18.946487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:18.946487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:18.946487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:18.946487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:18.946487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:18.946487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:18.946936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:18.946936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:18.946936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.946936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:18.947423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.947423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:18.947423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:18.947423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:18.947423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:18.947423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:18.947423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.947423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:18.947423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:18.947423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.947423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:18.947423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:18.947423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.947423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:18.947423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:18.947423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.947423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:30:18.948404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:30:18.948404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:18.948404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:18.948404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:18.948404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:18.948404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:18.948404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:18.948404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:18.948404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:18.948404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:18.948404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:18.948404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:18.948404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:18.948404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:18.948404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:18.948404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:18.948404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:18.948404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:18.948404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:18.949217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:18.949217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:18.949217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.949217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:18.954912 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:18.955829 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:18.955946 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:18.956265 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:18.957535 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:18.958063 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:18.958419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.959403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:18.959403 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:18.959403 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:18.959403 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:18.959403 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:18.959403 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:18.959403 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.959403 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:18.960424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.960424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:18.960424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:18.960424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:18.960424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.960424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:30:18.960424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:18.960424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:18.960424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:18.960424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:18.960424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:18.960424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:18.961261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:18.961261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:18.961261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:18.961261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:18.961261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:18.961261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:18.961261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:18.961261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:18.961261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:18.961261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:18.961261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:18.961261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:18.961261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:18.962058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.962058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:18.962058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.962058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:18.962058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:18.962058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:18.962058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:18.962058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:18.962058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.962058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:18.962058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:18.962058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.962058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:18.962058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:18.962058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.962920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:18.962920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:18.962920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:18.962920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:30:18.962920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:18.962920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:18.962920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:18.962920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:18.962920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:18.962920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:18.962920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:18.962920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:18.962920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:18.962920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:18.962920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:18.962920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:18.962920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:18.962920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:18.962920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:18.963910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:18.963910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:18.963910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:18.963910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:18.963910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:18.963910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:23.648133 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:23.648950 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:23.648950 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:23.650219 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:23.694915 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:23.695290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.695820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:23.695820 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:23.695820 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:23.695820 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:23.695820 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:23.695820 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:23.695820 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.695820 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:23.696893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.696893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:23.696893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:23.696893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:23.696893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.696893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:30:23.696893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:23.696893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:23.696893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:23.696893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:23.696893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:23.696893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:23.698394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:23.698394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:23.698394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:23.698394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:23.698394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:23.698394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:23.698394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:23.698394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:23.698394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:23.698394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:23.698394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:23.698394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:23.698394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:23.699312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.699312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:23.699312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.699312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:23.699312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:23.699312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:23.699312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:23.699312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:23.699312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.699312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:23.699312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:23.699312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.699312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:23.699312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:23.699312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.700017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:23.700017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:23.700017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.700017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:30:23.700017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:23.700017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:23.700017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:23.700017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:23.700017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:23.700017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:23.700017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:23.700017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:23.700017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:23.700017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:23.700017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:23.700017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:23.700017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:23.700017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:23.700017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:23.700825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:23.700825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:23.700825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:23.700825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:23.700825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.700825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:23.881392 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:23.881720 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:23.881847 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:23.882725 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:23.883033 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:23.883225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.883889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:23.883889 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:23.883889 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:23.883889 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:23.883889 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:23.883889 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:23.883889 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.883889 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:23.885147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.885147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:23.885147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:23.885147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:23.885147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.885147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:30:23.885147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:23.885147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:23.885147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:23.885147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:23.885147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:23.885147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:23.885755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:23.885755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:23.885755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:23.885755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:23.885755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:23.885755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:23.885755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:23.885755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:23.885755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:23.885755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:23.885755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:23.885755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:23.885755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:23.886341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.886341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:23.886341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.886341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:23.886341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:23.886341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:23.886341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:23.886341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:23.886341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.886341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:23.886341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:23.886341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.886341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:23.886341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:23.886341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.887490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:23.887490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:23.887490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.887490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:30:23.887490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:23.887490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:23.887490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:23.887490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:23.887490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:23.887490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:23.887490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:23.887490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:23.887490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:23.887490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:23.887490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:23.887490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:23.887490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:23.887490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:23.887490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:23.889652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:23.889652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:23.889652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:23.889652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:23.889652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.889652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:23.934337 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:23.934877 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:23.935080 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:23.935287 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:30:23.936081 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:23.936311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.936994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:23.936994 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:23.936994 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:23.936994 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:23.936994 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:23.936994 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:23.936994 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.936994 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:23.938346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.938346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:30:23.938346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:23.938346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:23.938346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:23.938346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:23.938346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:23.938346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:23.938346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:23.938346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:23.938346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:23.938346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:30:23.938346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:23.938346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:30:23.938346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:30:23.939224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:23.939224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:30:23.939224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:23.939224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:30:23.939224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:30:23.939224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:23.939224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:30:23.939224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.939224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:30:23.939774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.939774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:23.939774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:23.939774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:23.939774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.939774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:30:23.939774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:30:23.939774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:30:23.939774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:23.939774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:23.939774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:23.939774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:23.940385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:23.940385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:23.940385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:23.940385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:23.940385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:23.940385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:23.940385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:23.940385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:23.940385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:23.940385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:23.940385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:23.940385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:23.940385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:23.940385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:23.941145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:23.941145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:23.941145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.941145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:23.941379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.941379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:23.941379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:23.941379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:23.941379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:23.941379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:23.941379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.941379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:23.941379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:23.941379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.941379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:23.941379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:23.941379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.941379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:23.941379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:23.941379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.942180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:30:23.942180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:30:23.942180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:30:23.942180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:23.942180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:23.942180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:23.942180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:23.942180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:23.942180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:23.942180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:23.942180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:23.942180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:23.942180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:23.942180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:23.942180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:23.942180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:23.942180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:23.942180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:23.943092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:23.943092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:23.943092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:23.943092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:23.943092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:23.943092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.943092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:23.943092 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:23.943500 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:23.943564 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:23.943802 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:23.944097 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:23.944291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.944764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:23.944764 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:23.944764 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:23.944764 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:23.944764 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:23.944764 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:23.944764 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.944764 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:23.945480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.945480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:30:23.945480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:23.945480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:23.945480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:23.945480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:23.945480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:23.945480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:23.945480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:23.945480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:23.945480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:23.945480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:30:23.945480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:23.945480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:30:23.945480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:30:23.945931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:23.945931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:30:23.945931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:23.945931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:30:23.945931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:30:23.945931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:23.945931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:30:23.945931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.945931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:30:23.946322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.946322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:23.946322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:23.946322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:23.946322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.946322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:30:23.946322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:30:23.946322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:23.946322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:23.946322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:23.946322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:23.946663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:23.946663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:23.946663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:23.946663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:23.946663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:23.946663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:23.946663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:23.946663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:23.946663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:23.946663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:23.946663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:23.946663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:23.946663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:23.946663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:23.947038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:23.947038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:23.947038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.947038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:23.947458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.947458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:23.947458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:23.947458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:23.947458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:23.947458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:23.947458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.947458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:23.947458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:23.947458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.947458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:23.947458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:23.947458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.947458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:23.947458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:23.947458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.947458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:30:23.948143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:30:23.948143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:23.948143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:23.948143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:23.948143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:23.948143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:23.948143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:23.948143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:23.948143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:23.948143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:23.948143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:23.948143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:23.948143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:23.948143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:23.948143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:23.948143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:23.948143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:23.948143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:23.948143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:23.948899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:23.948899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:23.948899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.948899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:23.955971 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:23.956262 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:23.956433 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:23.956643 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:23.957359 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:23.957861 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:23.958143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.958967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:23.958967 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:23.958967 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:23.958967 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:23.958967 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:23.958967 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:23.958967 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.958967 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:23.960280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.960280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:23.960280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:23.960280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:23.960280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.960280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:30:23.960280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:23.960280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:23.960280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:23.960280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:23.960280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:23.960280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:23.961067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:23.961067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:23.961067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:23.961067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:23.961067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:23.961067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:23.961067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:23.961067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:23.961067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:23.961067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:23.961067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:23.961067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:23.961067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:23.961630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.961630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:23.961922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.961922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:23.961922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:23.961922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:23.961922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:23.961922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:23.961922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.961922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:23.961922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:23.961922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.961922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:23.961922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:23.961922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.961922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:23.961922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:23.961922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:23.962909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:30:23.962909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:23.962909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:23.962909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:23.962909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:23.962909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:23.962909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:23.962909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:23.962909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:23.962909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:23.962909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:23.962909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:23.962909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:23.962909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:23.962909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:23.962909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:23.962909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:23.962909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:23.962909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:23.963813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:23.963813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:23.963813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:28.648661 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:28.649379 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:28.649379 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:28.649923 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:28.679749 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:28.679948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.683066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:28.683066 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:28.683066 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:28.683066 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:28.683066 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:28.683066 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:28.683066 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.683066 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:28.684089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.684089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:28.684089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:28.684089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:28.684089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.684089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:30:28.684089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:28.684089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:28.684089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:28.684089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:28.684089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:28.684089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:28.684872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:28.684872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:28.684872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:28.684872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:28.684872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:28.684872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:28.684872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:28.684872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:28.684872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:28.684872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:28.684872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:28.684872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:28.684872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:28.685404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.685404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:28.685404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.685404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:28.685404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:28.685404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:28.685404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:28.685404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:28.685404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.685404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:28.685404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:28.685404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.685404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:28.685404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:28.685404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.685910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:28.685910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:28.685910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.685910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:30:28.685910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:28.685910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:28.685910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:28.685910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:28.685910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:28.685910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:28.685910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:28.685910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:28.685910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:28.685910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:28.685910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:28.685910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:28.685910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:28.685910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:28.685910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:28.686538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:28.686538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:28.686538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:28.686538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:28.686538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.686538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:28.884752 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:28.884752 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:28.884752 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:28.884752 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:28.884752 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:28.884752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.885244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:28.885244 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:28.885244 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:28.885244 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:28.885244 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:28.885244 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:28.885244 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.885244 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:28.885244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.885244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:28.885244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:28.885244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:28.885244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.885244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:30:28.885244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:28.885940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:28.885940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:28.885940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:28.885940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:28.885940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:28.885940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:28.885940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:28.885940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:28.885940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:28.885940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:28.885940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:28.885940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:28.885940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:28.885940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:28.886493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:28.886493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:28.886493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:28.886493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:28.886493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.886493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:28.886493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.886493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:28.886493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:28.886493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:28.886493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:28.886493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:28.886493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.886493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:28.887138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:28.887138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.887138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:28.887138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:28.887138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.887138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:28.887138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:28.887138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.887138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:30:28.887138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:28.887138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:28.887138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:28.887138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:28.887138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:28.887138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:28.887138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:28.887138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:28.887138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:28.887138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:28.887138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:28.891440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:28.891440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:28.891440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:28.891440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:28.891440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:28.891440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:28.891440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:28.891440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:28.891440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.891440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:28.935081 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:28.935406 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:28.935555 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:28.935555 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:30:28.936163 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:28.936399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.937015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:28.937015 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:28.937015 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:28.937015 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:28.937015 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:28.937015 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:28.937015 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.937015 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:28.937814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.937814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:30:28.937814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:28.937814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:28.937814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:28.937814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:28.937814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:28.937814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:28.937814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:28.937814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:28.937814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:28.937814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:30:28.937814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:28.937814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:30:28.937814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:30:28.939006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:28.939006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:30:28.939006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:28.939006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:30:28.939006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:30:28.939006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:28.939006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:30:28.939006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.939006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:30:28.939006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.939006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:28.939006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:28.939006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:28.939006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.939883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:30:28.939883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:30:28.939883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:30:28.939883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:28.939883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:28.939883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:28.939883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:28.939883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:28.939883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:28.939883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:28.939883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:28.939883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:28.939883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:28.940712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:28.940712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:28.940712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:28.940712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:28.940712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:28.940712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:28.940712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:28.940712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:28.940712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:28.940712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:28.940712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.940712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:28.941458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.941458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:28.941458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:28.941458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:28.941458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:28.941458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:28.941458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.941458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:28.941458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:28.941458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.941458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:28.941458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:28.941458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.941458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:28.941458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:28.941458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.942026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:30:28.942026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:30:28.942026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:30:28.942026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:28.942026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:28.942026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:28.942026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:28.942026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:28.942026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:28.942026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:28.942026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:28.942026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:28.942026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:28.942026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:28.942026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:28.942026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:28.942026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:28.942026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:28.942567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:28.942567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:28.942567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:28.942567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:28.942567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:28.942567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.942567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:28.943591 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:28.943980 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:28.944060 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:28.944164 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:28.944564 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:28.944848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.945353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:28.945353 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:28.945353 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:28.945353 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:28.945353 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:28.945353 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:28.945353 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.945353 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:28.946082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.946082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:30:28.946082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:28.946082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:28.946082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:28.946082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:28.946082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:28.946082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:28.946082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:28.946082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:28.946082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:28.946082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:30:28.946082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:28.946082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:30:28.946082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:30:28.946657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:28.946657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:30:28.946657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:28.946657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:30:28.946657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:30:28.946657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:28.946657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:30:28.946657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.946657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:30:28.946844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.946844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:28.946844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:28.946844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:28.946844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.946844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:30:28.946844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:30:28.946844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:28.946844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:28.946844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:28.946844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:28.947071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:28.947071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:28.947071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:28.947071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:28.947071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:28.947071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:28.947071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:28.947071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:28.947071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:28.947071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:28.947071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:28.947071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:28.947071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:28.947071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:28.947314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:28.947314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:28.947314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.947314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:28.947579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.947579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:28.947579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:28.947579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:28.947579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:28.947579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:28.947579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.947579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:28.947579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:28.947579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.947579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:28.947579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:28.947579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.947579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:28.947579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:28.947579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.947579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:30:28.948567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:30:28.948567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:28.948567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:28.948567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:28.948567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:28.948567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:28.948567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:28.948567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:28.948567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:28.948567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:28.948567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:28.948567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:28.948567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:28.948567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:28.948567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:28.948567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:28.948567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:28.948567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:28.948567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:28.949182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:28.949182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:28.949182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.949182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:28.956456 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:28.956898 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:28.956994 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:28.957185 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:28.957842 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:28.958246 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:28.958471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.959232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:28.959232 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:28.959232 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:28.959232 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:28.959232 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:28.959232 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:28.959232 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.959232 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:28.960148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.960148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:28.960148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:28.960148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:28.960148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.960148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:30:28.960148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:28.960148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:28.960148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:28.960148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:28.960148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:28.960148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:28.960570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:28.960570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:28.960570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:28.960570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:28.960570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:28.960570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:28.960570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:28.960570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:28.960570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:28.960570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:28.960570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:28.960570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:28.960570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:28.961775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.961775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:28.961775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.961775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:28.961775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:28.961775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:28.961775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:28.961775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:28.961775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.961775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:28.961775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:28.961775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.961775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:28.961775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:28.961775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.962192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:28.962192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:28.962192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:28.962192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:30:28.962192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:28.962192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:28.962192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:28.962192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:28.962192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:28.962192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:28.962192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:28.962192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:28.962192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:28.962192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:28.962192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:28.962192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:28.962192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:28.962192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:28.962192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:28.962724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:28.962724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:28.962724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:28.962724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:28.962724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:28.962724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:33.649281 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:33.649891 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:33.649891 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:33.650434 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:33.678132 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:33.678404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.679013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:33.679013 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:33.679013 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:33.679013 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:33.679013 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:33.679013 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:33.679013 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.679013 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:33.679778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.679778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:33.679778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:33.679778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:33.679778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.679778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:30:33.679778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:33.679778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:33.679778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:33.679778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:33.679778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:33.679778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:33.680422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:33.680422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:33.680422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:33.680422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:33.680422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:33.680422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:33.680422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:33.680422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:33.680422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:33.680422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:33.680422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:33.680422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:33.680422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:33.681011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.681011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:33.681011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.681011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:33.681011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:33.681011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:33.681011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:33.681011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:33.681011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.681011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:33.681011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:33.681011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.681011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:33.681011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:33.681011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.681643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:33.681643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:33.681643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.681643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:30:33.681643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:33.681643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:33.681643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:33.681643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:33.681643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:33.681643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:33.681643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:33.681643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:33.681643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:33.681643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:33.681643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:33.681643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:33.681643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:33.681643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:33.681643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:33.682441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:33.682441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:33.682441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:33.682441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:33.682441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.682441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:33.882866 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:33.883335 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:33.883448 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:33.884129 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:33.884305 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:33.884427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.885228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:33.885228 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:33.885228 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:33.885228 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:33.885228 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:33.885228 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:33.885228 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.885228 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:33.886217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.886217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:33.886217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:33.886217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:33.886217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.886217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:30:33.886217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:33.886217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:33.886217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:33.886217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:33.886217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:33.886217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:33.887585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:33.887585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:33.887585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:33.887585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:33.887585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:33.887585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:33.887585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:33.887585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:33.887585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:33.887585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:33.887585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:33.887585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:33.887585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:33.888208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.888208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:33.888208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.888208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:33.888208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:33.888208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:33.888208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:33.888208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:33.888208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.888208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:33.888208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:33.888208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.888208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:33.888208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:33.888208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.889021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:33.889021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:33.889021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.889021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:30:33.889021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:33.889021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:33.889021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:33.889021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:33.889021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:33.889021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:33.889021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:33.889021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:33.889021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:33.889021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:33.889021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:33.889021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:33.889021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:33.889021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:33.889021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:33.890394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:33.890394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:33.890394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:33.890394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:33.890394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.890394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:33.935745 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:33.936154 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:33.936238 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:33.936337 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:30:33.936813 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:33.936994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.937419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:33.937419 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:33.937419 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:33.937419 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:33.937419 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:33.937419 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:33.937419 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.937419 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:33.938106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.938106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:30:33.938106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:33.938106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:33.938106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:33.938106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:33.938106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:33.938106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:33.938106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:33.938106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:33.938106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:33.938106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:30:33.938106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:33.938106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:30:33.938106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:30:33.938622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:33.938622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:30:33.938622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:33.938622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:30:33.938622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:30:33.938622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:33.938622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:30:33.938622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.938622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:30:33.938904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.938904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:33.938904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:33.938904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:33.938904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.938904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:30:33.938904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:30:33.938904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:30:33.938904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:33.938904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:33.938904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:33.938904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:33.939348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:33.939348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:33.939348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:33.939348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:33.939348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:33.939348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:33.939348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:33.939348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:33.939348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:33.939348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:33.939348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:33.939348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:33.939348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:33.939348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:33.939818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:33.939818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:33.939818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.939818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:33.939818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.939818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:33.939818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:33.939818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:33.939818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:33.939818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:33.939818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.939818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:33.939818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:33.939818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.940286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:33.940286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:33.940286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.940286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:33.940286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:33.940286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.940286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:30:33.940286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:30:33.940286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:30:33.940286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:33.940286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:33.940286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:33.940286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:33.940286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:33.940286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:33.940286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:33.940286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:33.940286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:33.940286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:33.942271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:33.942271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:33.942271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:33.942271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:33.942271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:33.942271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:33.942271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:33.942271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:33.942271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:33.942271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:33.942271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.942271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:33.944301 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:33.944558 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:33.944652 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:33.944763 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:33.945230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:33.946055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.946055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:33.946055 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:33.946055 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:33.946055 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:33.946055 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:33.946055 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:33.946055 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.946055 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:33.946563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.946563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:30:33.946563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:33.946563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:33.946563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:33.946563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:33.946563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:33.946563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:33.946563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:33.946563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:33.946563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:33.946563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:30:33.946563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:33.946563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:30:33.946563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:30:33.947234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:33.947234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:30:33.947234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:33.947234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:30:33.947234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:30:33.947234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:33.947234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:30:33.947234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.947234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:30:33.947234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.947234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:33.947234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:33.947234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:33.947234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.947771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:30:33.947771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:30:33.947771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:33.947771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:33.947771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:33.947771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:33.947771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:33.947771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:33.947771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:33.947771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:33.947771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:33.947771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:33.947771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:33.948207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:33.948207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:33.948207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:33.948207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:33.948207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:33.948207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:33.948207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:33.948207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:33.948207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:33.948207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.948207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:33.948207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.948462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:33.948462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:33.948462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:33.948462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:33.948462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:33.948462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.948462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:33.948462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:33.948462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.948462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:33.948462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:33.948462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.948462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:33.948462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:33.948462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.948462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:30:33.948462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:30:33.948462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:33.948462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:33.948873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:33.948873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:33.948873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:33.948873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:33.948873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:33.948873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:33.948873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:33.948873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:33.948873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:33.948873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:33.948873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:33.948873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:33.948873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:33.948873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:33.948873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:33.948873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:33.948873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:33.948873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:33.948873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.948873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:33.956959 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:33.957352 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:33.957460 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:33.957586 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:33.958060 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:33.958305 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:33.958510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.959066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:33.959066 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:33.959066 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:33.959066 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:33.959066 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:33.959066 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:33.959066 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.959066 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:33.959938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.959938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:33.959938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:33.959938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:33.959938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.959938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:30:33.959938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:33.959938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:33.959938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:33.959938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:33.959938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:33.959938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:33.960865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:33.960865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:33.960865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:33.960865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:33.960865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:33.960865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:33.960865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:33.960865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:33.960865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:33.960865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:33.960865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:33.960865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:33.960865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:33.961763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.961763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:33.961763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.961763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:33.961763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:33.961763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:33.961763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:33.961763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:33.961763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.961763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:33.961763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:33.961763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.961763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:33.961763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:33.961763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.962420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:33.962420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:33.962420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:33.962420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:30:33.962420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:33.962420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:33.962420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:33.962420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:33.962420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:33.962420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:33.962420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:33.962420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:33.962420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:33.962420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:33.962420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:33.962420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:33.962420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:33.962420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:33.962420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:33.963523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:33.963523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:33.963523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:33.963523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:33.963523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:33.963523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:38.649930 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:38.651467 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:38.651467 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:38.652096 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:38.688506 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:38.688815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.689570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:38.689570 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:38.689570 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:38.689570 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:38.689570 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:38.689570 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:38.689570 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.689570 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:38.690702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.690702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:38.690702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:38.690702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:38.690702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.690702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:30:38.690702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:38.690702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:38.690702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:38.690702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:38.690702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:38.690702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:38.691314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:38.691314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:38.691314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:38.691314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:38.691314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:38.691314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:38.691314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:38.691314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:38.691314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:38.691314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:38.691314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:38.691314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:38.691314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:38.691799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.691799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:38.691976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.691976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:38.691976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:38.691976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:38.691976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:38.691976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:38.691976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.691976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:38.691976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:38.691976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.691976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:38.691976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:38.691976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.691976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:38.691976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:38.691976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.692543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:30:38.692543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:38.692543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:38.692543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:38.692543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:38.692543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:38.692543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:38.692543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:38.692543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:38.692543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:38.692543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:38.692543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:38.692543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:38.692543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:38.692543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:38.692543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:38.692543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:38.692543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:38.692543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:38.693210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:38.693210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.693210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:38.883415 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:38.884016 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:38.884143 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:38.885005 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:38.885236 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:38.885377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.886113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:38.886113 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:38.886113 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:38.886113 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:38.886113 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:38.886113 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:38.886113 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.886113 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:38.887244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.887244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:38.887244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:38.887244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:38.887244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.887244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:30:38.887244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:38.887244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:38.887244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:38.887244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:38.887244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:38.887244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:38.887573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:38.887573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:38.887573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:38.887573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:38.887573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:38.887573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:38.887573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:38.887573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:38.887573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:38.887573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:38.887573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:38.887573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:38.887573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:38.888362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.888362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:38.888441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.888441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:38.888441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:38.888441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:38.888441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:38.888441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:38.888441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.888441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:38.888441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:38.888441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.888441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:38.888441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:38.888441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.888441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:38.888441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:38.888441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.888829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:30:38.888829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:38.888829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:38.888829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:38.888829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:38.888829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:38.888829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:38.888829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:38.888829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:38.888829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:38.888829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:38.888829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:38.888829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:38.888829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:38.888829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:38.888829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:38.888829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:38.888829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:38.888829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:38.889293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:38.889293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.889293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:38.936280 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:38.936934 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:38.937153 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:38.937355 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:30:38.938057 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:38.938198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.939964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:38.939964 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:38.939964 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:38.939964 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:38.939964 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:38.939964 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:38.939964 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.939964 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:38.940812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.940812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:30:38.940812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:38.940812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:38.940812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:38.940812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:38.940812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:38.940812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:38.940812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:38.940812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:38.940812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:38.940812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:30:38.940812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:38.940812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:30:38.940812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:30:38.941648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:38.941648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:30:38.941648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:38.941648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:30:38.941648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:30:38.941648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:38.941648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:30:38.941648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.941648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:30:38.941648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.941648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:38.941648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:38.941648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:38.941648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.942379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:30:38.942379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:30:38.942379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:30:38.942379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:38.942379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:38.942379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:38.942379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:38.942379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:38.942379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:38.942379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:38.942379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:38.942379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:38.942379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:38.942975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:38.942975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:38.942975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:38.942975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:38.942975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:38.942975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:38.942975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:38.942975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:38.942975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:38.942975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:38.942975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.942975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:38.943583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.943583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:38.943583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:38.943583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:38.943583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:38.943583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:38.943583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.943583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:38.943583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:38.943583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.943583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:38.943583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:38.943583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.943583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:38.943583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:38.943583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.945521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:30:38.945521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:30:38.945521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:30:38.945521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:38.945521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:38.945521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:38.945521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:38.945521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:38.945521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:38.945521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:38.945521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:38.945521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:38.945521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:38.945521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:38.945521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:38.945521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:38.945521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:38.945521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:38.946611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:38.946611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:38.946611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:38.946611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:38.946611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:38.946611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.946611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:38.946611 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:38.946611 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:38.946611 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:38.947183 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:38.947183 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:38.947183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.947183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:38.947183 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:38.947183 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:38.947183 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:38.947183 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:38.947183 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:38.947183 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.947183 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:38.947832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.947832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:30:38.947832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:38.947832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:38.947832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:38.947832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:38.947832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:38.947832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:38.947832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:38.947832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:38.947832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:38.947832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:30:38.947832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:38.947832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:30:38.947832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:30:38.948343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:38.948343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:30:38.948343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:38.948343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:30:38.948343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:30:38.948343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:38.948343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:30:38.948343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.948343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:30:38.949299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.949299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:38.949299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:38.949299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:38.949299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.949299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:30:38.949299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:30:38.949299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:38.949299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:38.949299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:38.949299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:38.949817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:38.949817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:38.949817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:38.949817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:38.949817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:38.949817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:38.949817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:38.949817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:38.949817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:38.949817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:38.949817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:38.949817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:38.949817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:38.949817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:38.950186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:38.950186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:38.950186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.950186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:38.951346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.951346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:38.951346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:38.951346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:38.951346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:38.951346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:38.951346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.951346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:38.951346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:38.951346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.951346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:38.951346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:38.951346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.951346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:38.951346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:38.951346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.951346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:30:38.952186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:30:38.952186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:38.952186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:38.952186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:38.952186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:38.952186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:38.952186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:38.952186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:38.952186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:38.952186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:38.952186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:38.952186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:38.952186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:38.952186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:38.952186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:38.952186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:38.952186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:38.952186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:38.952186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:38.952697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:38.952697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:38.952697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.952697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:38.957365 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:38.957776 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:38.957890 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:38.958077 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:38.959554 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:38.959797 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:38.959944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.960549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:38.960549 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:38.960549 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:38.960549 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:38.960549 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:38.960549 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:38.960549 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.960549 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:38.961505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.961505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:38.961505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:38.961505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:38.961505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.961505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:30:38.961505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:38.961505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:38.961505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:38.961505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:38.961505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:38.961505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:38.961814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:38.961814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:38.961814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:38.961814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:38.961814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:38.961814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:38.961814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:38.961814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:38.961814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:38.961814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:38.961814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:38.961814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:38.961814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:38.962060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.962060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:38.962518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.962518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:38.962518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:38.962518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:38.962518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:38.962518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:38.962518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.962518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:38.962518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:38.962518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.962518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:38.962518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:38.962518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.962518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:38.962518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:38.962518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:38.962980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:30:38.962980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:38.962980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:38.962980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:38.962980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:38.962980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:38.962980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:38.962980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:38.962980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:38.962980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:38.962980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:38.962980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:38.962980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:38.962980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:38.962980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:38.962980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:38.962980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:38.962980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:38.962980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:38.963508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:38.963508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:38.963508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:43.651736 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:43.652245 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:43.652245 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:43.653605 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:43.692209 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:43.692519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.693268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:43.693268 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:43.693268 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:43.693268 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:43.693268 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:43.693268 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:43.693268 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.693268 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:43.694794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.694794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:43.694794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:43.694794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:43.694794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.694794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:30:43.694794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:43.694794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:43.694794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:43.694794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:43.694794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:43.694794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:43.695164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:43.695164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:43.695164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:43.695164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:43.695164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:43.695164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:43.695164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:43.695164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:43.695164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:43.695164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:43.695164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:43.695164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:43.695164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:43.695716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.695716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:43.696869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.696869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:43.696869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:43.696869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:43.696869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:43.696869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:43.696869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.696869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:43.696869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:43.696869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.696869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:43.696869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:43.696869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.696869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:43.696869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:43.696869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.697321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:30:43.697321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:43.697321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:43.697321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:43.697321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:43.697321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:43.697321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:43.697321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:43.697321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:43.697321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:43.697321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:43.697321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:43.697321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:43.697321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:43.697321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:43.697321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:43.697321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:43.697321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:43.697321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:43.697837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:43.697837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.697837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:43.884432 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:43.885096 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:43.885171 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:43.886055 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:43.886299 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:43.886477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.887343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:43.887343 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:43.887343 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:43.887343 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:43.887343 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:43.887343 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:43.887343 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.887343 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:43.889870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.889870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:43.889870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:43.889870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:43.889870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.889870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:30:43.889870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:43.889870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:43.889870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:43.889870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:43.889870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:43.889870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:43.890458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:43.890458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:43.890458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:43.890458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:43.890458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:43.890458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:43.890458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:43.890458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:43.890458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:43.890458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:43.890458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:43.890458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:43.890458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:43.891070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.891070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:43.891638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.891638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:43.891638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:43.891638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:43.891638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:43.891638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:43.891638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.891638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:43.891638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:43.891638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.891638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:43.891638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:43.891638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.891638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:43.891638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:43.891638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.892431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:30:43.892431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:43.892431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:43.892431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:43.892431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:43.892431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:43.892431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:43.892431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:43.892431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:43.892431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:43.892431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:43.892431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:43.892431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:43.892431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:43.892431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:43.892431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:43.892431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:43.892431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:43.892431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:43.893353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:43.893353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.893353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:43.937075 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:43.937818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:43.938111 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:43.938409 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:30:43.939082 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:43.939344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.940247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:43.940247 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:43.940247 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:43.940247 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:43.940247 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:43.940247 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:43.940247 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.940247 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:43.942099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.942099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:30:43.942099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:43.942099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:43.942099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:43.942099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:43.942099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:43.942099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:43.942099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:43.942099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:43.942099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:43.942099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:30:43.942099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:43.942099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:30:43.942099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:30:43.942625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:43.942625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:30:43.942625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:43.942625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:30:43.942625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:30:43.942625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:43.942625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:30:43.942625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.942625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:30:43.944630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.944630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:43.944630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:43.944630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:43.944630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.944630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:30:43.944630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:30:43.944630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:30:43.944630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:43.944630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:43.944630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:43.944630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:43.944999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:43.944999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:43.944999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:43.944999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:43.944999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:43.944999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:43.944999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:43.944999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:43.944999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:43.944999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:43.944999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:43.944999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:43.944999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:43.944999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:43.945342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:43.945342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:43.945342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.945342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:43.946332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.946332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:43.946332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:43.946332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:43.946332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:43.946332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:43.946332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.946332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:43.946332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:43.946332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.946332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:43.946332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:43.946332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.946332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:43.946332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:43.946332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.946784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:30:43.946784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:30:43.946784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:30:43.946784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:43.946784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:43.946784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:43.946784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:43.946784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:43.946784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:43.946784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:43.946784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:43.946784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:43.946784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:43.946784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:43.946784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:43.946784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:43.946784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:43.946784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:43.947255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:43.947255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:43.947255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:43.947255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:43.947255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:43.947255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.947255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:43.947255 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:43.947255 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:43.947255 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:43.947528 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:43.948447 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:43.948865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.949587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:43.949587 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:43.949587 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:43.949587 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:43.949587 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:43.949587 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:43.949587 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.949587 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:43.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:30:43.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:43.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:43.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:43.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:43.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:43.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:43.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:43.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:43.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:43.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:30:43.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:43.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:30:43.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:30:43.950841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:43.950841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:30:43.950841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:43.950841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:30:43.950841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:30:43.950841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:43.950841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:30:43.950841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.950841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:30:43.951309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.951309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:43.951309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:43.951309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:43.951309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.951309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:30:43.951309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:30:43.951309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:43.951309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:43.951309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:43.951309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:43.951603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:43.951603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:43.951603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:43.951603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:43.951603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:43.951603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:43.951603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:43.951603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:43.951603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:43.951603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:43.951603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:43.951603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:43.951603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:43.951603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:43.951955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:43.951955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:43.951955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.951955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:43.952479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.952479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:43.952479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:43.952479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:43.952479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:43.952479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:43.952479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.952479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:43.952479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:43.952479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.952479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:43.952479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:43.952479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.952479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:43.952479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:43.952479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.952479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:30:43.952932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:30:43.952932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:43.952932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:43.952932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:43.952932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:43.952932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:43.952932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:43.952932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:43.952932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:43.952932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:43.952932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:43.952932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:43.952932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:43.952932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:43.952932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:43.952932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:43.952932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:43.952932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:43.952932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:43.953407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:43.953407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:43.953407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.953407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:43.957966 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:43.958378 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:43.958526 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:43.958746 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:43.959138 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:43.959393 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:43.959545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.960176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:43.960176 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:43.960176 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:43.960176 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:43.960176 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:43.960176 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:43.960176 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.960176 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:43.961376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.961376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:43.961376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:43.961376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:43.961376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.961376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:30:43.961376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:43.961376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:43.961376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:43.961376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:43.961376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:43.961376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:43.961739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:43.961739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:43.961739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:43.961739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:43.961739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:43.961739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:43.961739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:43.961739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:43.961739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:43.961739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:43.961739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:43.961739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:43.961739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:43.962064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.962064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:43.962431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.962431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:43.962431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:43.962431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:43.962431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:43.962431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:43.962431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.962431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:43.962431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:43.962431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.962431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:43.962431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:43.962431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.962431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:43.962431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:43.962431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:43.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:30:43.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:43.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:43.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:43.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:43.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:43.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:43.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:43.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:43.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:43.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:43.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:43.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:43.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:43.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:43.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:43.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:43.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:43.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:43.963415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:43.963415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:43.963415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:48.652299 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:48.653097 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:48.653097 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:48.654426 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:48.689228 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:48.689407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.689938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:48.689938 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:48.689938 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:48.689938 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:48.689938 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:48.689938 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:48.689938 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.689938 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:48.691037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.691037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:48.691037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:48.691037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:48.691037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.691037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:30:48.691037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:48.691037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:48.691037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:48.691037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:48.691037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:48.691037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:48.691597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:48.691597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:48.691597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:48.691597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:48.691597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:48.691597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:48.691597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:48.691597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:48.691597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:48.691597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:48.691597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:48.691597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:48.691597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:48.692181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.692181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:48.692181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.692181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:48.692181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:48.692181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:48.692181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:48.692181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:48.692181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.692181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:48.692181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:48.692181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.692181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:48.692181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:48.692181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.692827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:48.692827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:48.692827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.692827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:30:48.692827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:48.692827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:48.692827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:48.692827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:48.692827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:48.692827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:48.692827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:48.692827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:48.692827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:48.692827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:48.692827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:48.692827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:48.692827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:48.692827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:48.692827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:48.693599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:48.693599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:48.693599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:48.693599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:48.693599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.693599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:48.885344 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:48.886134 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:48.886517 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:48.887617 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:48.887960 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:48.888198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.890922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:48.890922 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:48.890922 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:48.890922 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:48.890922 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:48.890922 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:48.890922 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.890922 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:48.891970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.891970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:48.891970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:48.891970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:48.891970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.891970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:30:48.891970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:48.891970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:48.891970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:48.891970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:48.891970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:48.891970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:48.892595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:48.892595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:48.892595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:48.892595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:48.892595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:48.892595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:48.892595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:48.892595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:48.892595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:48.892595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:48.892595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:48.892595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:48.892595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:48.893483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.893483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:48.893483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.893483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:48.893483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:48.893483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:48.893483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:48.893483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:48.893483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.893483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:48.893483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:48.893483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.893483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:48.893483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:48.893483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.894616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:48.894616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:48.894616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.894616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:30:48.894616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:48.894616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:48.894616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:48.894616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:48.894616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:48.894616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:48.894616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:48.894616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:48.894616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:48.894616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:48.894616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:48.894616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:48.894616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:48.894616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:48.894616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:48.895808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:48.895808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:48.895808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:48.895808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:48.895808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.895808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:48.937745 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:48.938381 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:48.938517 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:48.938837 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:30:48.939513 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:48.942717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.942717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:48.942717 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:48.942717 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:48.942717 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:48.942717 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:48.942717 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:48.942717 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.942717 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:48.942717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.942717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:30:48.942717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:48.942717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:48.942717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:48.942717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:48.942717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:48.942717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:48.943339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:48.943339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:48.943339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:48.943339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:30:48.943339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:48.943339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:30:48.943339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:30:48.943339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:48.943339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:30:48.943339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:48.943339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:30:48.943339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:30:48.943339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:48.943339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:30:48.943339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.943339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:30:48.943998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.943998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:48.943998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:48.943998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:48.943998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.943998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:30:48.943998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:30:48.943998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:30:48.943998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:48.943998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:48.943998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:48.943998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:48.944473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:48.944473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:48.944473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:48.944473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:48.944473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:48.944473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:48.944473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:48.944473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:48.944473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:48.944473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:48.944473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:48.944473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:48.944473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:48.944473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:48.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:48.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:48.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:48.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:48.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:48.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:48.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:48.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:48.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:48.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:48.945552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.946521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:48.946521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:48.946521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.946521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:48.946521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:48.946521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.946521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:30:48.946521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:30:48.946521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:30:48.946521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:48.946521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:48.946521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:48.946521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:48.946521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:48.946521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:48.946521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:48.946521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:48.946521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:48.946521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:48.947117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:48.947117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:48.947117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:48.947117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:48.947117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:48.947117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:48.947117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:48.947117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:48.947117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:48.947117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:48.947117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.947117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:48.947460 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:48.947654 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:48.947654 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:48.947801 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:48.948242 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:48.948467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.948915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:48.948915 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:48.948915 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:48.948915 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:48.948915 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:48.948915 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:48.948915 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.948915 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:48.949628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.949628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:30:48.949628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:48.949628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:48.949628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:48.949628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:48.949628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:48.949628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:48.949628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:48.949628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:48.949628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:48.949628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:30:48.949628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:48.949628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:30:48.949628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:30:48.950145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:48.950145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:30:48.950145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:48.950145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:30:48.950145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:30:48.950145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:48.950145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:30:48.950145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.950145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:30:48.950522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.950522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:48.950522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:48.950522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:48.950522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.950522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:30:48.950522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:30:48.950522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:48.950522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:48.950522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:48.950522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:48.950793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:48.950793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:48.950793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:48.950793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:48.950793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:48.950793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:48.950793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:48.950793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:48.950793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:48.950793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:48.950793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:48.950793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:48.950793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:48.950793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:48.951076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:48.951076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:48.951076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.951076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:48.951583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.951583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:48.951583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:48.951583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:48.951583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:48.951583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:48.951583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.951583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:48.951583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:48.951583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.951583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:48.951583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:48.951583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.951583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:48.951583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:48.951583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.951583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:30:48.952031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:30:48.952031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:48.952031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:48.952031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:48.952031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:48.952031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:48.952031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:48.952031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:48.952031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:48.952031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:48.952031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:48.952031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:48.952031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:48.952031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:48.952031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:48.952031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:48.952031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:48.952031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:48.952031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:48.952519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:48.952519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:48.952519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.952519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:48.958473 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:48.958786 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:48.958884 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:48.959041 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:48.959784 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:48.960171 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:48.960413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.961134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:48.961134 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:48.961134 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:48.961134 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:48.961134 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:48.961134 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:48.961134 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.961134 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:48.962170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.962170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:48.962170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:48.962170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:48.962170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.962170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:30:48.962170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:48.962170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:48.962170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:48.962170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:48.962170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:48.962170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:48.962526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:48.962526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:48.962526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:48.962526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:48.962526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:48.962526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:48.962526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:48.962526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:48.962526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:48.962526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:48.962526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:48.962526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:48.962526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:48.962936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.962936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:48.963406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.963406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:48.963406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:48.963406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:48.963406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:48.963406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:48.963406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.963406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:48.963406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:48.963406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.963406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:48.963406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:48.963406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.963406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:48.963406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:48.963406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:48.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:30:48.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:48.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:48.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:48.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:48.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:48.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:48.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:48.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:48.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:48.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:48.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:48.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:48.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:48.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:48.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:48.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:48.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:48.963994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:48.964505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:48.964505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:48.964505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:53.652970 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:53.653595 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:53.653595 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:53.654313 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:53.692531 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:53.692531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.693220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:53.693220 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:53.693220 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:53.693220 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:53.693220 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:53.693220 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:53.693220 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.693220 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:53.694419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.694419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:53.694419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:53.694419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:53.694419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.694419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:30:53.694419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:53.694419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:53.694419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:53.694419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:53.694419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:53.694419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:53.694822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:53.694822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:53.694822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:53.694822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:53.694822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:53.694822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:53.694822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:53.694822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:53.694822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:53.694822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:53.694822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:53.694822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:53.694822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:53.695058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.695058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:53.695511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.695511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:53.695511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:53.695511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:53.695511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:53.695511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:53.695511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.695511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:53.695511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:53.695511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.695511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:53.695511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:53.695511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.695511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:53.695511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:53.695511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.695842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:30:53.695842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:53.695842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:53.695842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:53.695842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:53.695842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:53.695842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:53.695842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:53.695842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:53.695842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:53.695842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:53.695842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:53.695842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:53.695842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:53.695842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:53.695842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:53.695842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:53.695842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:53.695842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:53.696208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:53.696208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.696208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:53.887056 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:53.887585 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:53.888034 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:53.889342 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:53.889706 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:53.890053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.890959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:53.890959 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:53.890959 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:53.890959 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:53.890959 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:53.890959 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:53.890959 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.890959 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:53.892559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.892559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:53.892559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:53.892559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:53.892559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.892559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:30:53.892559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:53.892559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:53.892559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:53.892559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:53.892559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:53.892559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:53.893210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:53.893210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:53.893210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:53.893210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:53.893210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:53.893210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:53.893210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:53.893210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:53.893210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:53.893210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:53.893210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:53.893210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:53.893210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:53.893828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.893828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:53.893828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.893828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:53.893828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:53.893828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:53.893828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:53.893828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:53.893828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.893828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:53.893828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:53.893828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.893828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:53.893828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:53.893828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.894998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:53.894998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:53.894998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.894998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:30:53.894998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:53.894998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:53.894998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:53.894998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:53.894998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:53.894998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:53.894998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:53.894998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:53.894998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:53.894998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:53.894998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:53.894998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:53.894998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:53.894998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:53.894998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:53.895486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:53.895486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:53.895486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:53.895486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:53.895486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.895486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:53.940213 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:53.940643 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:53.940876 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:53.941070 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:30:53.941766 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:53.942007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.942742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:53.942742 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:53.942742 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:53.942742 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:53.942742 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:53.942742 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:53.942742 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.942742 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:53.944165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.944165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:30:53.944165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:53.944165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:53.944165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:53.944165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:53.944165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:53.944165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:53.944165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:53.944165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:53.944165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:53.944165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:30:53.944165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:53.944165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:30:53.944165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:30:53.944820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:53.944820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:30:53.944820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:53.944820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:30:53.944820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:30:53.944820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:53.944820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:30:53.944820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.944820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:30:53.945621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.945621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:53.945621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:53.945621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:53.945621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.945621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:30:53.945621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:30:53.945621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:30:53.945621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:53.945621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:53.945621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:53.945621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:53.946136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:53.946136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:53.946136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:53.946136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:53.946136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:53.946136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:53.946136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:53.946136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:53.946136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:53.946136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:53.946136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:53.946136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:53.946136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:53.946136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:53.946526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:53.946526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:53.946526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.946526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:53.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:53.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:53.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:53.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:53.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:53.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:53.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:53.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:53.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:53.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:53.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:53.947225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.947745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:30:53.947745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:30:53.947745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:30:53.947745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:53.947745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:53.947745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:53.947745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:53.947745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:53.947745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:53.947745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:53.947745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:53.947745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:53.947745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:53.947745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:53.947745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:53.947745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:53.947745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:53.947745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:53.948168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:53.948168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:53.948168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:53.948168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:53.948168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:53.948168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.948168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:53.948168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:53.948168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:53.948168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:53.948440 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:53.948440 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:53.948901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.949635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:53.949635 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:53.949635 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:53.949635 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:53.949635 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:53.949635 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:53.949635 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.949635 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:53.950430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.950430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:30:53.950430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:53.950430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:53.950430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:53.950430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:53.950430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:53.950430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:53.950430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:53.950430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:53.950430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:53.950430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:30:53.950430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:53.950430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:30:53.950430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:30:53.950875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:53.950875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:30:53.950875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:53.950875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:30:53.950875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:30:53.950875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:53.950875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:30:53.950875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.950875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:30:53.951285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.951285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:53.951285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:53.951285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:53.951285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.951285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:30:53.951285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:30:53.951285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:53.951285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:53.951285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:53.951285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:53.951637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:53.951637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:53.951637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:53.951637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:53.951637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:53.951637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:53.951637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:53.951637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:53.951637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:53.951637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:53.951637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:53.951637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:53.951637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:53.951637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:53.952054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:53.952054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:53.952054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.952054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:53.952318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.952318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:53.952318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:53.952318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:53.952318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:53.952318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:53.952318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.952318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:53.952318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:53.952318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.952318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:53.952318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:53.952318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.952318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:53.952318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:53.952318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.952318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:30:53.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:30:53.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:53.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:53.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:53.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:53.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:53.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:53.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:53.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:53.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:53.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:53.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:53.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:53.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:53.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:53.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:53.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:53.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:53.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:53.953243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:53.953243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:53.953243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.953243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:53.958932 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:53.959399 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:53.959399 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:53.959399 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:53.962381 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:53.962632 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:53.962829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.963434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:53.963434 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:53.963434 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:53.963434 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:53.963434 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:53.963434 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:53.963434 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.963434 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:53.964242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.964242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:53.964242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:53.964242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:53.964242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.964242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:30:53.964242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:53.964242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:53.964242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:53.964242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:53.964242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:53.964242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:53.964770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:53.964770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:53.964770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:53.964770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:53.964770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:53.964770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:53.964770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:53.964770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:53.964770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:53.964770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:53.964770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:53.964770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:53.964770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:53.965221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.965221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:53.965221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.965221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:53.965221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:53.965221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:53.965221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:53.965221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:53.965221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.965221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:53.965221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:53.965221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.965221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:53.965221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:53.965221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.965777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:53.965777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:53.965777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:53.965777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:30:53.965777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:53.965777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:53.965777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:53.965777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:53.965777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:53.965777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:53.965777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:53.965777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:53.965777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:53.965777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:53.965777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:53.965777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:53.965777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:53.965777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:53.965777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:53.966488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:53.966488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:53.966488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:53.966488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:53.966488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:53.966488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:58.653937 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:58.654765 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:58.654765 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:58.655605 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:58.687062 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:58.687245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.687802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:58.687802 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:58.687802 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:58.687802 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:58.687802 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:58.687802 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:58.687802 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.687802 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:58.688518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.688518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:58.688518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:58.688518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:58.688518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.688518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:30:58.688518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:58.688518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:58.688518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:58.688518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:58.688518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:58.688518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:58.688893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:58.688893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:58.688893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:58.688893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:58.688893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:58.688893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:58.688893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:58.688893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:58.688893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:58.688893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:58.688893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:58.688893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:58.688893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:58.689250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.689250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:58.689250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.689250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:58.689250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:58.689250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:58.689250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:58.689250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:58.689250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.689250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:58.689250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:58.689250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.689250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:58.689250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:58.689250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.689878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:58.689878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:58.689878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.689878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:30:58.689878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:58.689878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:58.689878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:58.689878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:58.689878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:58.689878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:58.689878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:58.689878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:58.689878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:58.689878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:58.689878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:58.689878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:58.689878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:58.689878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:58.689878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:58.690642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:58.690642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:58.690642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:58.690642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:58.690642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.690642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:58.887759 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:58.888230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:58.888358 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:58.889299 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:58.889561 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:58.889790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.890485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:58.890485 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:58.890485 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:58.890485 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:58.890485 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:58.890485 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:58.890485 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.890485 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:58.891488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.891488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:58.891488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:58.891488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:58.891488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.891488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:30:58.891488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:58.891488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:58.891488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:58.891488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:58.891488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:58.891488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:58.892116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:58.892116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:58.892116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:58.892116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:58.892116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:58.892116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:58.892116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:58.892116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:58.892116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:58.892116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:58.892116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:58.892116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:58.892116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:58.892773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.892773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:58.892773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.892773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:58.892773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:58.892773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:58.892773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:58.892773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:58.892773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.892773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:58.892773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:58.892773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.892773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:58.892773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:58.892773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.893556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:58.893556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:58.893556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.893556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:30:58.893556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:58.893556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:58.893556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:58.893556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:58.893556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:58.893556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:58.893556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:58.893556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:58.893556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:58.893556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:58.893556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:58.893556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:58.893556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:58.893556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:58.893556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:58.894447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:58.894447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:58.894447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:58.894447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:58.894447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.894447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:58.941042 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:58.941362 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:58.941436 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:58.941555 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:30:58.942312 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:58.942535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.943108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:58.943108 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:58.943108 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:58.943108 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:58.943108 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:58.943108 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:58.943108 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.943108 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:58.943782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.943782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:30:58.943782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:58.943782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:58.943782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:58.943782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:58.943782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:58.943782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:58.943782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:58.943782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:58.943782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:58.943782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:30:58.943782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:58.943782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:30:58.943782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:30:58.944458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:58.944458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:30:58.944458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:58.944458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:30:58.944458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:30:58.944458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:58.944458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:30:58.944458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.944458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:30:58.944458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.944458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:58.944458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:58.944458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:58.944458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.945240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:30:58.945240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:30:58.945240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:30:58.945240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:58.945240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:58.945240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:58.945240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:58.945240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:58.945240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:58.945240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:58.945240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:58.945240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:58.945240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:58.945851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:58.945851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:58.945851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:58.945851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:58.945851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:58.945851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:58.945851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:58.945851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:58.945851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:58.945851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:58.945851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.945851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:58.946360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.946360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:58.946360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:58.946360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:58.946360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:58.946360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:58.946360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.946360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:58.946360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:58.946360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.946360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:58.946360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:58.946360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.946360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:58.946360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:58.946360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.947038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:30:58.947038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:30:58.947038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:30:58.947038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:58.947038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:58.947038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:58.947038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:58.947038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:58.947038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:58.947038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:58.947038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:58.947038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:58.947038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:58.947038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:58.947038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:58.947038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:58.947038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:58.947038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:58.947815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:58.947815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:58.947815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:58.947815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:58.947815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:58.947815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.947815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:58.948100 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:58.948404 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:58.948404 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:58.948487 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:58.948933 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:58.949176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.949930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:58.949930 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:58.949930 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:58.949930 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:58.949930 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:58.949930 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:58.949930 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.949930 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:58.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:30:58.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:58.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:58.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:58.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:58.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:58.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:58.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:58.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:30:58.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:58.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:30:58.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:58.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:30:58.950434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:30:58.950848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:58.950848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:30:58.950848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:58.950848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:30:58.950848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:30:58.950848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:58.950848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:30:58.950848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.950848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:30:58.951098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.951098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:58.951098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:58.951098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:58.951098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.951098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:30:58.951098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:30:58.951098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:58.951098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:58.951098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:58.951098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:58.951445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:58.951445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:58.951445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:58.951445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:58.951445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:58.951445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:58.951445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:58.951445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:58.951445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:58.951445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:58.951445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:58.951445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:58.951445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:58.951445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:58.951808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:58.951808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:58.951808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.951808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:58.951948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.951948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:58.951948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:58.951948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:58.951948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:58.951948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:58.951948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.951948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:58.951948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:58.951948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.951948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:58.951948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:58.951948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.951948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:58.951948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:58.951948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.951948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:30:58.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:30:58.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:30:58.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:30:58.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:58.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:58.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:58.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:58.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:58.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:58.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:58.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:58.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:58.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:58.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:58.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:58.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:58.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:58.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:58.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:58.952972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:58.952972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:58.952972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.952972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:30:58.959398 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:30:58.959735 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:30:58.959870 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:30:58.960109 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:30:58.960826 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:58.961159 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:30:58.961313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.961949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:30:58.961949 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:58.961949 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:30:58.961949 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:30:58.961949 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:58.961949 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:30:58.961949 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.961949 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:30:58.963199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.963199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:30:58.963199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:58.963199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:30:58.963199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.963199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:30:58.963199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:58.963199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:58.963199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:58.963199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:58.963199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:58.963199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:58.963456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:30:58.963456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:58.963456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:30:58.963456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:58.963456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:30:58.963456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:30:58.963456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:58.963456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:30:58.963456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:58.963456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:30:58.963456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:30:58.963456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:58.963456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:30:58.963714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.963714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:30:58.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:30:58.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:30:58.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:30:58.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:30:58.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:30:58.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:30:58.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:30:58.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:30:58.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:30:58.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:30:58.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:30:58.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:30:58.965269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:30:58.965269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:30:58.965269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:58.965269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:58.965269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:30:58.965269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:30:58.965269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:30:58.965269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:30:58.965269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:30:58.965269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:30:58.965269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:30:58.965269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:30:58.965269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:30:58.965269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:30:58.965269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:30:58.965269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:30:58.965269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:30:58.965269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:30:58.965269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:30:58.965735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:30:58.965735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:30:58.965735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:03.654570 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:03.655295 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:03.655816 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:03.656886 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:03.658179 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=240 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:31:03.866136 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:03.866136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.866601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:03.866601 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:03.866601 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:03.866601 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:03.866601 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:03.866601 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:03.866601 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.866601 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:03.867632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.211 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.867632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:03.867632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:03.867632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:03.867632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.867632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:31:03.867632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:03.867632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:03.867632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:03.867632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:03.867632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:03.867632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:03.868445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:03.868445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:03.868445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:03.868445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:03.868445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:03.868445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:03.868445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:03.868445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:03.868445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:03.868445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:03.868445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:03.868445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:03.868445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:03.869153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.869153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:03.869153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.869153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:03.869153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:03.869153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:03.869153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:03.869153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:03.869153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.869153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:03.869153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:03.869153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.869153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:03.869153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:03.869153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.870030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:03.870030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:03.870030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.870030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:31:03.870030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:03.870030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:03.870030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:03.870030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:03.870030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:03.870030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:03.870030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:03.870030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:03.870030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:03.870030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:03.870030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:03.870030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:03.870030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:03.870030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:03.870030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:03.872065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:03.872065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:03.872065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:03.872065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:03.872065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.872065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:03.888312 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:03.888861 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:03.889063 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:03.889961 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:03.890210 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:03.891630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.891630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:03.891630 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:03.891630 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:03.891630 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:03.891630 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:03.891630 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:03.891630 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.891630 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:03.892489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.892489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:03.892489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:03.892489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:03.892489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.892489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:31:03.892489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:03.892489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:03.892489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:03.892489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:03.892489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:03.892489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:03.892975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:03.892975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:03.892975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:03.892975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:03.892975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:03.892975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:03.892975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:03.892975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:03.892975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:03.892975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:03.892975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:03.892975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:03.892975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:03.893331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.893331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:03.894279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.894279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:03.894279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:03.894279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:03.894279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:03.894279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:03.894279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.894279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:03.894279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:03.894279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.894279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:03.894279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:03.894279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.894279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:03.894279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:03.894279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.894805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:31:03.894805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:03.894805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:03.894805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:03.894805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:03.894805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:03.894805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:03.894805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:03.894805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:03.894805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:03.894805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:03.894805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:03.894805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:03.894805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:03.894805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:03.894805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:03.894805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:03.894805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:03.894805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:03.895317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:03.895317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.895317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:03.941590 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:03.942638 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:03.942638 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:03.942638 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:31:03.943243 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:03.943683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.944812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:03.944812 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:03.944812 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:03.944812 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:03.944812 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:03.944812 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:03.944812 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.944812 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:03.946098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.946098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:31:03.946098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:03.946098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:03.946098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:03.946098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:03.946098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:03.946098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:03.946098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:03.946098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:03.946098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:03.946098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:31:03.946098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:03.946098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:31:03.946098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:31:03.946564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:03.946564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:31:03.946564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:03.946564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:31:03.946564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:31:03.946564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:03.946564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:31:03.946564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.946564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:31:03.947019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.947019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:03.947019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:03.947019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:03.947019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.947019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:31:03.947019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:31:03.947019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:31:03.947019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:03.947019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:03.947019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:03.947019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:03.947456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:03.947456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:03.947456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:03.947456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:03.947456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:03.947456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:03.947456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:03.947456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:03.947456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:03.947456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:03.947456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:03.947456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:03.947456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:03.947456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:03.947874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:03.947874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:03.947874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.947874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:03.947986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.947986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:03.947986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:03.947986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:03.947986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:03.947986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:03.947986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.947986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:03.947986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:03.947986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.947986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:03.947986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:03.947986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.947986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:03.947986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:03.947986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.948402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:31:03.948402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:31:03.948402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:31:03.948402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:03.948402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:03.948402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:03.948402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:03.948402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:03.948402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:03.948402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:03.948402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:03.948402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:03.948402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:03.948402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:03.948402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:03.948402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:03.948402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:03.948402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:03.949016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:03.949016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:03.949016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:03.949016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:03.949016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:03.949016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.949016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:03.949016 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:03.949016 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:03.949016 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:03.949298 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:03.949298 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:03.949626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.951220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:03.951220 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:03.951220 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:03.951220 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:03.951220 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:03.951220 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:03.951220 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.951220 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:03.951220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.951220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:31:03.951220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:03.951220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:03.951220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:03.951220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:03.951220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:03.951220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:03.951220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:03.951220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:03.951887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:03.951887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:31:03.951887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:03.951887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:31:03.951887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:31:03.951887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:03.951887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:31:03.951887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:03.951887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:31:03.951887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:31:03.951887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:03.951887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:31:03.951887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.951887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:31:03.951887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.952302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:03.952302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:03.952302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:03.952302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.952302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:31:03.952302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:31:03.952302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:03.952302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:03.952302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:03.952302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:03.952302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:03.952302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:03.952302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:03.952840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:03.952840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:03.952840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:03.952840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:03.952840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:03.952840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:03.952840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:03.952840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:03.952840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:03.952840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:03.952840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:03.952840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:03.952840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:03.953233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.953233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:03.953233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.953233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:03.953233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:03.953233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:03.953233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:03.953233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:03.953233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.953233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:03.953233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:03.953233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.953233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:03.953233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:03.953233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.953786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:03.953786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:03.953786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.953786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:31:03.953786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:31:03.953786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:03.953786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:03.953786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:03.953786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:03.953786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:03.953786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:03.953786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:03.953786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:03.953786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:03.953786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:03.953786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:03.953786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:03.953786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:03.953786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:03.954275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:03.954275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:03.954275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:03.954275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:03.954275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:03.954275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:03.954275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.954275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:03.960025 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:03.960336 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:03.960423 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:03.960612 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:03.961112 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:03.961685 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:03.961904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.962653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:03.962653 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:03.962653 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:03.962653 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:03.962653 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:03.962653 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:03.962653 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.962653 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:03.963696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.963696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:03.963696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:03.963696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:03.963696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.963696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:31:03.963696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:03.963696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:03.963696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:03.963696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:03.963696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:03.963696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:03.964169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:03.964169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:03.964169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:03.964169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:03.964169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:03.964169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:03.964169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:03.964169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:03.964169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:03.964169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:03.964169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:03.964169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:03.964169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:03.964509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.964509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:03.964753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.964753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:03.964753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:03.964753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:03.964753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:03.964753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:03.964753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.964753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:03.964753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:03.964753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.964753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:03.964753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:03.964753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.964753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:03.964753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:03.964753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:03.965185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:31:03.965185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:03.965185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:03.965185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:03.965185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:03.965185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:03.965185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:03.965185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:03.965185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:03.965185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:03.965185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:03.965185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:03.965185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:03.965185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:03.965185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:03.965185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:03.965185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:03.965185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:03.965185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:03.965822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:03.965822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:03.965822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:08.655323 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:08.657239 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:08.657239 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:08.657546 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:08.699956 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:08.702257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.703135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:08.703135 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:08.703135 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:08.703135 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:08.703135 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:08.703135 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:08.703135 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.703135 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:08.704316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.704316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:08.704316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:08.704316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:08.704316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.704316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:31:08.704316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:08.704316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:08.704316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:08.704316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:08.704316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:08.704316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:08.704736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:08.704736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:08.704736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:08.704736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:08.704736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:08.704736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:08.704736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:08.704736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:08.704736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:08.704736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:08.704736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:08.704736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:08.704736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:08.705124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.705124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:08.705542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.705542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:08.705542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:08.705542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:08.705542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:08.705542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:08.705542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.705542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:08.705542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:08.705542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.705542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:08.705542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:08.705542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.705542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:08.705542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:08.705542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.706057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:31:08.706057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:08.706057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:08.706057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:08.706057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:08.706057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:08.706057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:08.706057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:08.706057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:08.706057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:08.706057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:08.706057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:08.706057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:08.706057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:08.706057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:08.706057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:08.706057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:08.706057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:08.706057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:08.706646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:08.706646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.706646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:08.889044 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:08.889938 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:08.889938 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:08.891024 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:08.891371 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:08.891729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.892773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:08.892773 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:08.892773 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:08.892773 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:08.892773 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:08.892773 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:08.892773 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.892773 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:08.894716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.894716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:08.894716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:08.894716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:08.894716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.894716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:31:08.894716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:08.894716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:08.894716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:08.894716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:08.894716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:08.894716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:08.895426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:08.895426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:08.895426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:08.895426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:08.895426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:08.895426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:08.895426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:08.895426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:08.895426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:08.895426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:08.895426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:08.895426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:08.895426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:08.896064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.896064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:08.897037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.897037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:08.897037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:08.897037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:08.897037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:08.897037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:08.897037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.897037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:08.897037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:08.897037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.897037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:08.897037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:08.897037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.897037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:08.897037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:08.897037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.897900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:31:08.897900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:08.897900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:08.897900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:08.897900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:08.897900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:08.897900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:08.897900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:08.897900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:08.897900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:08.897900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:08.897900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:08.897900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:08.897900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:08.897900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:08.897900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:08.897900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:08.897900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:08.897900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:08.898763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:08.898763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.898763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:08.942323 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:08.942842 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:08.943092 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:08.943333 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:31:08.943985 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:08.944400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.945281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:08.945281 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:08.945281 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:08.945281 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:08.945281 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:08.945281 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:08.945281 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.945281 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:08.946651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.946651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:31:08.946651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:08.946651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:08.946651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:08.946651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:08.946651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:08.946651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:08.946651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:08.946651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:08.946651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:08.946651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:31:08.946651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:08.946651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:31:08.946651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:31:08.948196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:08.948196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:31:08.948196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:08.948196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:31:08.948196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:31:08.948196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:08.948196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:31:08.948196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.948196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:31:08.949028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.949028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:08.949028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:08.949028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:08.949028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.949028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:31:08.949028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:31:08.949028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:31:08.949028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:08.949028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:08.949028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:08.949028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:08.949768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:08.949768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:08.949768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:08.949768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:08.949768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:08.949768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:08.949768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:08.949768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:08.949768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:08.949768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:08.949768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:08.949768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:08.949768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:08.949768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:08.950408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:08.950408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:08.950408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.950408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:08.950614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.950614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:08.950614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:08.950614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:08.950614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:08.950614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:08.950614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.950614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:08.950614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:08.950614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.950614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:08.950614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:08.950614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.950614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:08.950614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:08.950614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.951090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:31:08.951090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:31:08.951090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:31:08.951090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:08.951090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:08.951090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:08.951090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:08.951090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:08.951090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:08.951090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:08.951090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:08.951090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:08.951090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:08.951090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:08.951090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:08.951090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:08.951090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:08.951090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:08.951763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:08.951763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:08.951763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:08.951763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:08.951763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:08.951763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.951763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:08.951763 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:08.951763 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:08.952205 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:08.952205 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:08.952471 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:08.952724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.953247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:08.953247 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:08.953247 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:08.953247 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:08.953247 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:08.953247 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:08.953247 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.953247 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:08.954133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.954133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:31:08.954133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:08.954133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:08.954133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:08.954133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:08.954133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:08.954133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:08.954133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:08.954133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:08.954133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:08.954133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:31:08.954133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:08.954133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:31:08.954133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:31:08.954808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:08.954808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:31:08.954808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:08.954808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:31:08.954808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:31:08.954808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:08.954808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:31:08.954808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.954808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:31:08.955393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.955393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:08.955393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:08.955393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:08.955393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.955393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:31:08.955393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:31:08.955393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:08.955393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:08.955393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:08.955393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:08.955938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:08.955938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:08.955938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:08.955938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:08.955938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:08.955938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:08.955938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:08.955938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:08.955938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:08.955938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:08.955938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:08.955938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:08.955938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:08.955938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:08.956509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:08.956509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:08.956509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.956509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:08.956974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.956974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:08.956974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:08.956974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:08.956974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:08.956974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:08.956974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.956974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:08.956974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:08.956974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.956974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:08.956974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:08.956974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.956974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:08.956974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:08.956974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.956974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:31:08.957487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:31:08.957487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:08.957487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:08.957487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:08.957487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:08.957487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:08.957487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:08.957487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:08.957487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:08.957487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:08.957487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:08.957487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:08.957487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:08.957487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:08.957487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:08.957487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:08.957487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:08.957487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:08.957487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:08.958018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:08.958018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:08.958018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.958018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:08.960507 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:08.960865 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:08.961014 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:08.961243 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:08.961695 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:08.962095 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:08.962420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.963362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:08.963362 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:08.963362 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:08.963362 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:08.963362 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:08.963362 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:08.963362 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.963362 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:08.964403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.964403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:08.964403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:08.964403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:08.964403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.964403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:31:08.964403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:08.964403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:08.964403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:08.964403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:08.964403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:08.964403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:08.964974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:08.964974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:08.964974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:08.964974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:08.964974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:08.964974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:08.964974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:08.964974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:08.964974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:08.964974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:08.964974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:08.964974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:08.964974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:08.965993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.965993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:08.965993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.965993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:08.965993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:08.965993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:08.965993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:08.965993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:08.965993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.965993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:08.965993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:08.965993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.965993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:08.965993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:08.965993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.966433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:08.966433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:08.966433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:08.966433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:31:08.966433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:08.966433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:08.966433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:08.966433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:08.966433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:08.966433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:08.966433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:08.966433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:08.966433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:08.966433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:08.966433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:08.966433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:08.966433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:08.966433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:08.966433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:08.966935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:08.966935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:08.966935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:08.966935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:08.966935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:08.966935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:13.055645 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:13.056483 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:13.056483 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:13.068071 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
Apr 28 20:31:13.068336 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=106506) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
Apr 28 20:31:13.068501 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 7ff08466-af6d-4e13-bb1d-8987170a5734 (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:31:13.102181 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-7ff08466-af6d-4e13-bb1d-8987170a5734, resource={'id': '7ff08466-af6d-4e13-bb1d-8987170a5734', 'name': 'private', 'tenant_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'project_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['4b461bd9-09ea-4f87-a007-a11ab022ff7f', '8f3677a2-e2be-443a-a90c-2cd7b3095c49'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-28T20:10:58Z', 'updated_at': '2026-04-28T20:11:02Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 46342}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.105083 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-7ff08466-af6d-4e13-bb1d-8987170a5734, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.105685 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-27fbc209-4b05-4b4a-8d5c-40d6d87b7e22, external_ids_update=None, columns={'tag': 46342}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.105980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.106866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:13.106866 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.106866 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:13.106866 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.106866 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.106866 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:13.106866 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.106866 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:13.107981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 7ff08466-af6d-4e13-bb1d-8987170a5734 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.107981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:13.107981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:31:13.107981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:31:13.107981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:31:13.107981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:31:13.107981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:31:13.107981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:13.107981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.107981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:13.107981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:13.108356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:13.108356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.108356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:13.108356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:13.108356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:13.108356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:13.108356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:13.108356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:13.108356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:13.108356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:13.108356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.108356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:13.108356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.108356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.108761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:13.108761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.108761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:13.108761 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 8a1fb774-5693-4139-8b75-6ab5273228c7 (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:31:13.134817 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, resource={'id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'name': 'public', 'tenant_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'project_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['335d12ce-2d0c-4086-b3f4-2af390c03d70', '5308c10f-28d1-441c-acb2-d1d6dc3d2ece'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-04-28T20:11:05Z', 'updated_at': '2026-04-28T20:11:13Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.135102 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.135530 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-7dbf713e-d963-4884-ae64-da4ce196e719, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.135840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.136422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:13.136422 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.136422 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:13.136422 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.136422 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.136422 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:13.136422 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.136422 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:13.137457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 8a1fb774-5693-4139-8b75-6ab5273228c7 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.137457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:13.137457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:31:13.137457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:31:13.137457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:31:13.137457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:31:13.137457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:31:13.137457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:13.137457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.137457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:13.137457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:13.137798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:13.137798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.137798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:13.137798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:13.137798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:13.137798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:13.137798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:13.137798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:13.137798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:13.137798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:13.137798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.137798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:13.137798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.137798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.139175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:13.139175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.139175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:13.139175 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource c6718637-daa9-4db1-bd11-63abe3891db6 (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:31:13.165173 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-c6718637-daa9-4db1-bd11-63abe3891db6, resource={'id': 'c6718637-daa9-4db1-bd11-63abe3891db6', 'name': 'admin_net', 'tenant_id': '3eb96b75a3044dd0a338616e1d9081d0', 'project_id': '3eb96b75a3044dd0a338616e1d9081d0', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['c1cc9883-a7e2-4abc-8892-a7c1b200d3ad'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-28T20:15:13Z', 'updated_at': '2026-04-28T20:15:14Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 7626}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.166515 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-c6718637-daa9-4db1-bd11-63abe3891db6, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.166515 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-7461d8df-a8a0-40c3-87a1-26ed88f9f7ce, external_ids_update=None, columns={'tag': 7626}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.166515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.167827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:13.167827 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.167827 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:13.167827 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.167827 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.167827 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:13.167827 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.167827 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:13.170255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource c6718637-daa9-4db1-bd11-63abe3891db6 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.170255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:13.170255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:31:13.170255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:31:13.170255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:31:13.170255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:31:13.170255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:31:13.170255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:13.170255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.170255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:13.170255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:13.170697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:13.170697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.170697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:13.170697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:13.170697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:13.170697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:13.170697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:13.170697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:13.170697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:13.170697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:13.170697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.170697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:13.170697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.170697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.171051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:13.171051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.171051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:13.171051 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource d3dfebbd-7943-4813-b509-739084ada63b (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:31:13.202772 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d3dfebbd-7943-4813-b509-739084ada63b, resource={'id': 'd3dfebbd-7943-4813-b509-739084ada63b', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-pi48i', 'tenant_id': 'a11fcd8844504e7eaf509ece683e8c8f', 'project_id': 'a11fcd8844504e7eaf509ece683e8c8f', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['c68bc19b-c88a-4cd2-bf3a-d847d267d23f'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-28T20:15:49Z', 'updated_at': '2026-04-28T20:15:50Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 58143}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.203381 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-d3dfebbd-7943-4813-b509-739084ada63b, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-pi48i', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.204130 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-ed33b0a1-b0b7-4e33-aa96-92c114a82f5d, external_ids_update=None, columns={'tag': 58143}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.204470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.205556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:13.205556 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.205556 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:13.205556 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.205556 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.205556 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:13.205556 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.205556 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:13.207178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource d3dfebbd-7943-4813-b509-739084ada63b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.207178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:13.207178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:31:13.207178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:31:13.207178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:31:13.207178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:31:13.207178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:31:13.207178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:13.207178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.207178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:13.207178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:13.207492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:13.207492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.207492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:13.207492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:13.207492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:13.207492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:13.207492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:13.207492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:13.207492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:13.207492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:13.207492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.207492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:13.207492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.207492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.208004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:13.208004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.208004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:13.208004 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 6feb526a-13c6-41ac-983d-ea1d2251ff3d (type: routers) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:31:13.293156 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, resource={'id': '6feb526a-13c6-41ac-983d-ea1d2251ff3d', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-pi48i', 'project_id': 'a11fcd8844504e7eaf509ece683e8c8f', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'external_fixed_ips': [{'subnet_id': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece', 'ip_address': '172.24.5.87'}, {'subnet_id': '335d12ce-2d0c-4086-b3f4-2af390c03d70', 'ip_address': '2001:db8::2d8'}], 'enable_snat': True}, 'gw_port_id': '73493630-b0ac-48ec-be98-5b048d2e2ce7', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-pi48i', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'external_fixed_ips': [{'ip_address': '172.24.5.87', 'subnet_id': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece'}, {'ip_address': '2001:db8::2d8', 'subnet_id': '335d12ce-2d0c-4086-b3f4-2af390c03d70'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-04-28T20:15:51Z', 'updated_at': '2026-04-28T20:15:54Z', 'revision_number': 3, 'tenant_id': 'a11fcd8844504e7eaf509ece683e8c8f'}, resource_type=routers, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.294046 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-pi48i', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.295374 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "73493630-b0ac-48ec-be98-5b048d2e2ce7"')], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.295970 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "73493630-b0ac-48ec-be98-5b048d2e2ce7"')], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.296253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.298013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:13.298013 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.298013 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:13.298013 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.298013 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.298013 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:13.298013 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.298013 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:13.298534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Unable to update router 6feb526a-13c6-41ac-983d-ea1d2251ff3d. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.300366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 6feb526a-13c6-41ac-983d-ea1d2251ff3d (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.300366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:13.300366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:31:13.300366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:31:13.300366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:31:13.300366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:31:13.300366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
Apr 28 20:31:13.300366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Apr 28 20:31:13.300366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
Apr 28 20:31:13.300366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Apr 28 20:31:13.300366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Apr 28 20:31:13.300708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Apr 28 20:31:13.300708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
Apr 28 20:31:13.300708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:13.300708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.300708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:13.300708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:13.300708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:13.300708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.300708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:13.300708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:13.300708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:13.300708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:13.300708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:13.300708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:13.301040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:13.301040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:13.301040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.301040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:13.301040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.301040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.301040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:13.301040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.301040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:13.301040 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 73493630-b0ac-48ec-be98-5b048d2e2ce7 (type: router_ports) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:31:13.409075 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-73493630-b0ac-48ec-be98-5b048d2e2ce7, resource={'id': '73493630-b0ac-48ec-be98-5b048d2e2ce7', 'name': '', 'network_id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'project_id': '', 'mac_address': 'fa:16:3e:f8:07:38', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6feb526a-13c6-41ac-983d-ea1d2251ff3d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece', 'ip_address': '172.24.5.87'}, {'subnet_id': '335d12ce-2d0c-4086-b3f4-2af390c03d70', 'ip_address': '2001:db8::2d8'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000171185', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-04-28T20:15:51Z', 'updated_at': '2026-04-28T20:15:54Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.409410 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-73493630-b0ac-48ec-be98-5b048d2e2ce7, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece 335d12ce-2d0c-4086-b3f4-2af390c03d70', 'neutron:network_name': 'neutron-8a1fb774-5693-4139-8b75-6ab5273228c7', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.87/24', '2001:db8::2d8/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.410089 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=73493630-b0ac-48ec-be98-5b048d2e2ce7, lrouter_port=lrp-73493630-b0ac-48ec-be98-5b048d2e2ce7, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.410670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.411632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:13.411632 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.411632 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:13.411632 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.411632 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.411632 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:13.411632 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.411632 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:13.412789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 73493630-b0ac-48ec-be98-5b048d2e2ce7 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.412789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:13.412789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:31:13.412789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:31:13.412789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:31:13.412789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:31:13.412789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1971, in update_router_port
Apr 28 20:31:13.412789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:13.412789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.412789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:13.412789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:13.413294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:13.413294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.413294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:13.413294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:13.413294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:13.413294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:13.413294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:13.413294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:13.413294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:13.413294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:13.413294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.413294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:13.413294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.413294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.413787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:13.413787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.413787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:13.413787 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_for_inconsistencies finished in 0.356 seconds
Apr 28 20:31:13.656166 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:13.657359 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:13.657359 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:13.658369 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:13.694231 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.694480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.695441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:13.695441 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.695441 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:13.695441 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.695441 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.695441 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:13.695441 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.695441 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:13.696176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.696176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:13.696176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:13.696176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:13.696176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.696176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:31:13.696176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:13.696176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.696176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:13.696176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:13.696176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:13.696176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.696548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:13.696548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:13.696548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:13.696548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:13.696548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:13.696548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:13.696548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:13.696548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:13.696548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.696548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:13.696548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.696548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.696548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:13.696800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.696800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:13.696934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.696934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:13.696934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:13.696934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:13.696934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:13.696934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:13.696934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.696934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:13.696934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:13.696934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.696934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:13.696934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:13.696934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.696934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:13.696934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:13.696934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.697245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:31:13.697245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:13.697245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.697245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:13.697245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:13.697245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:13.697245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.697245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:13.697245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:13.697245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:13.697245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:13.697245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:13.697245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:13.697245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:13.697245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:13.697245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.697245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:13.697245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.697245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.697731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:13.697731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.697731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:13.889861 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:13.890362 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:13.890502 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:13.891602 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.891936 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.892179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.892966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:13.892966 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.892966 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:13.892966 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.892966 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.892966 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:13.892966 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.892966 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:13.893985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.893985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:13.893985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:13.893985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:13.893985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.893985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:31:13.893985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:13.893985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.893985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:13.893985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:13.893985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:13.893985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.894850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:13.894850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:13.894850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:13.894850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:13.894850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:13.894850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:13.894850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:13.894850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:13.894850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.894850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:13.894850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.894850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.894850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:13.895549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.895549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:13.895549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.895549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:13.895549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:13.895549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:13.895549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:13.895549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:13.895549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.895549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:13.895549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:13.895549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.895549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:13.895549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:13.895549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.896491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:13.896491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:13.896491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.896491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:31:13.896491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:13.896491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.896491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:13.896491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:13.896491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:13.896491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.896491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:13.896491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:13.896491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:13.896491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:13.896491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:13.896491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:13.896491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:13.896491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:13.896491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.897486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:13.897486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.897486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.897486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:13.897486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.897486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:13.943272 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:13.943905 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:13.944113 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:13.944376 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:31:13.945371 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.947672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.948793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:13.948793 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.948793 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:13.948793 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.948793 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.948793 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:13.948793 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.948793 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:13.950103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.950103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:31:13.950103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:13.950103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:13.950103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.950103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:13.950103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:13.950103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:13.950103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.950103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:13.950103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:13.950103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:31:13.950103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:13.950103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:31:13.950103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:31:13.950501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:13.950501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:31:13.950501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.950501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:31:13.950501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.950501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.950501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:31:13.950501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.950501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:31:13.951524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.951524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:13.951524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:13.951524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:13.951524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.951524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:31:13.951524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:31:13.951524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:31:13.951524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:13.951524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:13.951524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.951524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:13.951893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:13.951893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:13.951893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.951893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:13.951893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:13.951893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:13.951893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:13.951893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:13.951893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:13.951893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:13.951893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:13.951893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.951893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:13.951893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.952270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.952270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:13.952270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.952270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:13.952270 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:13.953001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.953001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:13.953001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:13.953001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:13.953001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:13.953001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:13.953001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.953001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:13.953001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:13.953001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.953001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:13.953001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:13.953001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.953001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:13.953001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:13.953001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.953418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:31:13.953418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:31:13.953418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:31:13.953418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:13.953418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:13.953418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.953418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:13.953418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:13.953418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:13.953418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.953418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:13.953418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:13.953418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:13.953418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:13.953418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:13.953418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:13.953418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:13.953418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:13.953890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.953890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:13.953890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.953890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.953890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:13.953890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.953890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:13.953890 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:13.953890 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:13.953890 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:13.954180 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.954180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.954649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:13.954649 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.954649 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:13.954649 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.954649 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.954649 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:13.954649 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.954649 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:13.955432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.955432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:31:13.955432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:13.955432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:13.955432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.955432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:13.955432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:13.955432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:13.955432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.955432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:13.955432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:13.955432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:31:13.955432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:13.955432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:31:13.955432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:31:13.955835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:13.955835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:31:13.955835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.955835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:31:13.955835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.955835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.955835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:31:13.955835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.955835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:31:13.956280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.956280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:13.956280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:13.956280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:13.956280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.956280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:31:13.956280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:31:13.956280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:13.956280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:13.956280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.956280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:13.956567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:13.956567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:13.956567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.956567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:13.956567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:13.956567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:13.956567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:13.956567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:13.956567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:13.956567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:13.956567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:13.956567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.956567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:13.956567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.957062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.957062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:13.957062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.957062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:13.957328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.957328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:13.957328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:13.957328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:13.957328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:13.957328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:13.957328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.957328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:13.957328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:13.957328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.957328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:13.957328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:13.957328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.957328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:13.957328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:13.957328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.957328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:31:13.957891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:31:13.957891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:13.957891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:13.957891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.957891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:13.957891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:13.957891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:13.957891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.957891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:13.957891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:13.957891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:13.957891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:13.957891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:13.957891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:13.957891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:13.957891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:13.957891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.957891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:13.957891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.958367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.958367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:13.958367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.958367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:13.960873 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:13.961114 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:13.961215 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:13.961376 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:13.961810 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.962065 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:13.962253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.962853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:13.962853 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.962853 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:13.962853 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.962853 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.962853 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:13.962853 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.962853 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:13.963668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.963668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:13.963668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:13.963668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:13.963668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.963668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:31:13.963668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:13.963668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.963668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:13.963668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:13.963668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:13.963668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.963980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:13.963980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:13.963980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:13.963980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:13.963980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:13.963980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:13.963980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:13.963980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:13.963980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.963980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:13.963980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.963980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.963980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:13.964306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.964306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:13.964745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.964745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:13.964745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:13.964745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:13.964745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:13.964745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:13.964745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.964745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:13.964745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:13.964745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.964745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:13.964745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:13.964745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.964745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:13.964745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:13.964745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:13.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:31:13.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:13.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:13.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:13.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:13.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:13.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:13.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:13.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:13.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:13.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:13.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:13.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:13.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:13.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:13.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:13.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:13.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:13.965573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:13.965573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:13.965573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:18.656927 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:18.657784 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:18.657784 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:18.659057 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:18.690857 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:18.691150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.692055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:18.692055 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:18.692055 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:18.692055 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:18.692055 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:18.692055 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:18.692055 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.692055 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:18.692956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.692956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:18.692956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:18.692956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:18.692956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.692956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:31:18.692956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:18.692956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:18.692956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:18.692956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:18.692956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:18.692956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:18.693722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:18.693722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:18.693722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:18.693722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:18.693722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:18.693722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:18.693722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:18.693722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:18.693722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:18.693722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:18.693722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:18.693722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:18.693722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:18.694422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.694422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:18.694422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.694422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:18.694422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:18.694422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:18.694422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:18.694422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:18.694422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.694422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:18.694422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:18.694422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.694422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:18.694422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:18.694422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.695223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:18.695223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:18.695223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.695223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:31:18.695223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:18.695223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:18.695223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:18.695223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:18.695223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:18.695223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:18.695223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:18.695223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:18.695223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:18.695223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:18.695223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:18.695223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:18.695223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:18.695223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:18.695223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:18.696321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:18.696321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:18.696321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:18.696321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:18.696321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.696321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:18.890859 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:18.891561 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:18.891785 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:18.892942 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:18.893290 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:18.893596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.894481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:18.894481 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:18.894481 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:18.894481 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:18.894481 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:18.894481 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:18.894481 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.894481 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:18.896387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.896387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:18.896387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:18.896387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:18.896387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.896387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:31:18.896387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:18.896387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:18.896387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:18.896387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:18.896387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:18.896387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:18.896758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:18.896758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:18.896758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:18.896758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:18.896758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:18.896758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:18.896758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:18.896758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:18.896758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:18.896758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:18.896758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:18.896758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:18.896758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:18.897099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.897099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:18.898391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.898391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:18.898391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:18.898391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:18.898391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:18.898391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:18.898391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.898391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:18.898391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:18.898391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.898391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:18.898391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:18.898391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.898391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:18.898391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:18.898391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.899036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:31:18.899036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:18.899036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:18.899036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:18.899036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:18.899036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:18.899036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:18.899036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:18.899036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:18.899036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:18.899036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:18.899036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:18.899036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:18.899036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:18.899036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:18.899036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:18.899036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:18.899036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:18.899036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:18.899515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:18.899515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.899515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:18.944232 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:18.944903 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:18.945108 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:18.945314 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:31:18.946066 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:18.946398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.947447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:18.947447 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:18.947447 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:18.947447 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:18.947447 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:18.947447 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:18.947447 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.947447 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:18.949445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.949445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:31:18.949445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:18.949445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:18.949445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:18.949445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:18.949445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:18.949445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:18.949445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:18.949445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:18.949445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:18.949445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:31:18.949445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:18.949445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:31:18.949445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:31:18.950265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:18.950265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:31:18.950265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:18.950265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:31:18.950265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:31:18.950265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:18.950265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:31:18.950265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.950265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:31:18.951167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.951167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:18.951167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:18.951167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:18.951167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.951167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:31:18.951167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:31:18.951167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:31:18.951167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:18.951167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:18.951167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:18.951167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:18.951840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:18.951840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:18.951840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:18.951840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:18.951840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:18.951840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:18.951840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:18.951840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:18.951840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:18.951840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:18.951840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:18.951840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:18.951840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:18.951840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:18.952423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:18.952423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:18.952423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.952423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:18.952775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.952775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:18.952775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:18.952775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:18.952775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:18.952775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:18.952775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.952775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:18.952775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:18.952775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.952775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:18.952775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:18.952775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.952775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:18.952775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:18.952775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.953562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:31:18.953562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:31:18.953562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:31:18.953562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:18.953562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:18.953562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:18.953562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:18.953562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:18.953562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:18.953562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:18.953562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:18.953562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:18.953562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:18.953562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:18.953562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:18.953562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:18.953562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:18.953562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:18.954418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:18.954418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:18.954418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:18.954418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:18.954418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:18.954418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.954418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:18.954418 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:18.954418 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:18.954418 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:18.954757 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:18.954757 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:18.954757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.955275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:18.955275 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:18.955275 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:18.955275 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:18.955275 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:18.955275 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:18.955275 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.955275 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:18.956046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.956046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:31:18.956046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:18.956046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:18.956046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:18.956046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:18.956046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:18.956046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:18.956046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:18.956046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:18.956046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:18.956046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:31:18.956046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:18.956046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:31:18.956046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:31:18.956572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:18.956572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:31:18.956572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:18.956572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:31:18.956572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:31:18.956572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:18.956572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:31:18.956572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.956572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:31:18.956948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.956948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:18.956948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:18.956948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:18.956948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.956948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:31:18.956948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:31:18.956948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:18.956948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:18.956948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:18.956948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:18.957390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:18.957390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:18.957390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:18.957390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:18.957390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:18.957390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:18.957390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:18.957390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:18.957390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:18.957390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:18.957390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:18.957390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:18.957390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:18.957390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:18.957791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:18.957791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:18.957791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.957791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:18.958060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.958060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:18.958060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:18.958060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:18.958060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:18.958060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:18.958060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.958060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:18.958060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:18.958060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.958060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:18.958060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:18.958060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.958060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:18.958060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:18.958060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.958060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:31:18.958858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:31:18.958858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:18.958858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:18.958858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:18.958858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:18.958858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:18.958858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:18.958858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:18.958858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:18.958858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:18.958858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:18.958858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:18.958858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:18.958858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:18.958858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:18.958858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:18.958858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:18.958858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:18.958858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:18.959498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:18.959498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:18.959498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.959498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:18.961290 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:18.961697 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:18.961831 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:18.961995 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:18.962436 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:18.962794 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:18.963010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.963669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:18.963669 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:18.963669 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:18.963669 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:18.963669 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:18.963669 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:18.963669 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.963669 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:18.964514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.964514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:18.964514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:18.964514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:18.964514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.964514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:31:18.964514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:18.964514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:18.964514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:18.964514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:18.964514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:18.964514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:18.964898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:18.964898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:18.964898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:18.964898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:18.964898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:18.964898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:18.964898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:18.964898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:18.964898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:18.964898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:18.964898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:18.964898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:18.964898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:18.965256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.965256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:18.965554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.965554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:18.965554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:18.965554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:18.965554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:18.965554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:18.965554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.965554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:18.965554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:18.965554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.965554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:18.965554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:18.965554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.965554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:18.965554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:18.965554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:18.966091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:31:18.966091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:18.966091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:18.966091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:18.966091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:18.966091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:18.966091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:18.966091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:18.966091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:18.966091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:18.966091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:18.966091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:18.966091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:18.966091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:18.966091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:18.966091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:18.966091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:18.966091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:18.966091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:18.966644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:18.966644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:18.966644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:23.657657 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:23.659230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:23.659230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:23.659230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:23.695307 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:23.695738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.696665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:23.696665 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:23.696665 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:23.696665 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:23.696665 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:23.696665 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:23.696665 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.696665 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:23.697628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.697628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:23.697628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:23.697628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:23.697628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.697628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:31:23.697628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:23.697628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:23.697628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:23.697628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:23.697628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:23.697628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:23.698035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:23.698035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:23.698035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:23.698035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:23.698035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:23.698035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:23.698035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:23.698035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:23.698035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:23.698035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:23.698035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:23.698035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:23.698035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:23.698411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.698411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:23.698481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.698481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:23.698481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:23.698481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:23.698481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:23.698481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:23.698481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.698481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:23.698481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:23.698481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.698481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:23.698481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:23.698481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.698481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:23.698481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:23.698481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.698979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:31:23.698979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:23.698979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:23.698979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:23.698979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:23.698979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:23.698979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:23.698979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:23.698979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:23.698979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:23.698979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:23.698979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:23.698979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:23.698979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:23.698979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:23.698979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:23.698979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:23.698979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:23.698979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:23.699461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:23.699461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.699461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:23.891523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:23.891891 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:23.892080 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:23.893023 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:23.893318 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:23.893509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.894220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:23.894220 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:23.894220 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:23.894220 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:23.894220 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:23.894220 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:23.894220 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.894220 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:23.896177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.896177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:23.896177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:23.896177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:23.896177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.896177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:31:23.896177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:23.896177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:23.896177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:23.896177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:23.896177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:23.896177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:23.896826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:23.896826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:23.896826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:23.896826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:23.896826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:23.896826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:23.896826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:23.896826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:23.896826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:23.896826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:23.896826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:23.896826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:23.896826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:23.897481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.897481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:23.897601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.897601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:23.897601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:23.897601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:23.897601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:23.897601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:23.897601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.897601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:23.897601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:23.897601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.897601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:23.897601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:23.897601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.897601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:23.897601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:23.897601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.898521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:31:23.898521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:23.898521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:23.898521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:23.898521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:23.898521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:23.898521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:23.898521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:23.898521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:23.898521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:23.898521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:23.898521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:23.898521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:23.898521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:23.898521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:23.898521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:23.898521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:23.898521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:23.898521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:23.899607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:23.899607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.899607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:23.945022 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:23.945622 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:23.945769 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:23.946011 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:31:23.947147 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:23.947147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.948248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:23.948248 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:23.948248 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:23.948248 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:23.948248 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:23.948248 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:23.948248 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.948248 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:23.949922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.949922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:31:23.949922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:23.949922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:23.949922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:23.949922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:23.949922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:23.949922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:23.949922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:23.949922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:23.949922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:23.949922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:31:23.949922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:23.949922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:31:23.949922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:31:23.950934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:23.950934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:31:23.950934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:23.950934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:31:23.950934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:31:23.950934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:23.950934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:31:23.950934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.950934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:31:23.951380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.951380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:23.951380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:23.951380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:23.951380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.951380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:31:23.951380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:31:23.951380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:31:23.951380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:23.951380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:23.951380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:23.951380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:23.952196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:23.952196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:23.952196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:23.952196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:23.952196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:23.952196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:23.952196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:23.952196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:23.952196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:23.952196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:23.952196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:23.952196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:23.952196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:23.952196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:23.953099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:23.953099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:23.953099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.953099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:23.953476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.953476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:23.953476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:23.953476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:23.953476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:23.953476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:23.953476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.953476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:23.953476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:23.953476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.953476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:23.953476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:23.953476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.953476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:23.953476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:23.953476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.954579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:31:23.954579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:31:23.954579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:31:23.954579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:23.954579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:23.954579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:23.954579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:23.954579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:23.954579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:23.954579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:23.954579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:23.954579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:23.954579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:23.954579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:23.954579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:23.954579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:23.954579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:23.954579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:23.955876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:23.955876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:23.955876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:23.955876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:23.955876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:23.955876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.955876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:23.955876 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:23.955876 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:23.955876 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:23.956650 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:23.956650 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:23.956650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.956650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:23.956650 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:23.956650 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:23.956650 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:23.956650 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:23.956650 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:23.956650 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.956650 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:23.957250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.957250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:31:23.957250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:23.957250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:23.957250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:23.957250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:23.957250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:23.957250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:23.957250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:23.957250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:23.957250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:23.957250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:31:23.957250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:23.957250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:31:23.957250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:31:23.957538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:23.957538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:31:23.957538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:23.957538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:31:23.957538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:31:23.957538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:23.957538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:31:23.957538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.957538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:31:23.958145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.958145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:23.958145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:23.958145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:23.958145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.958145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:31:23.958145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:31:23.958145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:23.958145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:23.958145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:23.958145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:23.958430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:23.958430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:23.958430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:23.958430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:23.958430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:23.958430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:23.958430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:23.958430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:23.958430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:23.958430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:23.958430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:23.958430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:23.958430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:23.958430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:23.958830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:23.958830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:23.958830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.958830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:23.959232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.959232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:23.959232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:23.959232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:23.959232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:23.959232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:23.959232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.959232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:23.959232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:23.959232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.959232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:23.959232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:23.959232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.959232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:23.959232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:23.959232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.959232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:31:23.959708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:31:23.959708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:23.959708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:23.959708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:23.959708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:23.959708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:23.959708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:23.959708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:23.959708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:23.959708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:23.959708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:23.959708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:23.959708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:23.959708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:23.959708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:23.959708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:23.959708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:23.959708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:23.959708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:23.960177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:23.960177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:23.960177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.960177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:23.961797 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:23.962044 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:23.962149 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:23.962296 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:23.962767 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:23.963141 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:23.963365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.964126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:23.964126 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:23.964126 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:23.964126 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:23.964126 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:23.964126 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:23.964126 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.964126 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:23.964982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.964982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:23.964982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:23.964982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:23.964982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.964982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:31:23.964982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:23.964982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:23.964982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:23.964982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:23.964982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:23.964982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:23.965428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:23.965428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:23.965428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:23.965428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:23.965428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:23.965428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:23.965428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:23.965428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:23.965428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:23.965428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:23.965428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:23.965428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:23.965428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:23.965875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.965875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:23.965991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.965991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:23.965991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:23.965991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:23.965991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:23.965991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:23.965991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.965991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:23.965991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:23.965991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.965991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:23.965991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:23.965991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.965991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:23.965991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:23.965991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:23.966439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:31:23.966439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:23.966439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:23.966439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:23.966439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:23.966439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:23.966439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:23.966439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:23.966439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:23.966439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:23.966439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:23.966439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:23.966439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:23.966439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:23.966439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:23.966439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:23.966439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:23.966439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:23.966439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:23.966998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:23.966998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:23.966998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:28.658351 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:28.659430 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:28.659430 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:28.660463 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:28.693523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:28.693929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.694855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:28.694855 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:28.694855 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:28.694855 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:28.694855 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:28.694855 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:28.694855 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.694855 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:28.695754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.695754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:28.695754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:28.695754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:28.695754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.695754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:31:28.695754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:28.695754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:28.695754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:28.695754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:28.695754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:28.695754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:28.696401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:28.696401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:28.696401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:28.696401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:28.696401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:28.696401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:28.696401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:28.696401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:28.696401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:28.696401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:28.696401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:28.696401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:28.696401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:28.697055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.697055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:28.697055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.697055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:28.697055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:28.697055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:28.697055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:28.697055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:28.697055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.697055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:28.697055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:28.697055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.697055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:28.697055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:28.697055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.697912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:28.697912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:28.697912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.697912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:31:28.697912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:28.697912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:28.697912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:28.697912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:28.697912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:28.697912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:28.697912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:28.697912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:28.697912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:28.697912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:28.697912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:28.697912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:28.697912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:28.697912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:28.697912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:28.700024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:28.700024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:28.700024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:28.700024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:28.700024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.700024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:28.892219 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:28.892813 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:28.892995 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:28.894014 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:28.894359 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:28.894737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.895710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:28.895710 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:28.895710 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:28.895710 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:28.895710 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:28.895710 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:28.895710 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.895710 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:28.897342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.897342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:28.897342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:28.897342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:28.897342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.897342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:31:28.897342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:28.897342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:28.897342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:28.897342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:28.897342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:28.897342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:28.899440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:28.899440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:28.899440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:28.899440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:28.899440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:28.899440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:28.899440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:28.899440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:28.899440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:28.899440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:28.899440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:28.899440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:28.899440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:28.900704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.900704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:28.900704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.900704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:28.900704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:28.900704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:28.900704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:28.900704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:28.900704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.900704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:28.900704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:28.900704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.900704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:28.900704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:28.900704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.901144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:28.901144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:28.901144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.901144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:31:28.901144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:28.901144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:28.901144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:28.901144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:28.901144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:28.901144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:28.901144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:28.901144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:28.901144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:28.901144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:28.901144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:28.901144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:28.901144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:28.901144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:28.901144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:28.901595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:28.901595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:28.901595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:28.901595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:28.901595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.901595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:28.945826 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:28.946159 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:28.946300 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:28.946499 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:31:28.947268 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:28.947633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.948715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:28.948715 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:28.948715 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:28.948715 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:28.948715 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:28.948715 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:28.948715 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.948715 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:28.949872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.949872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:31:28.949872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:28.949872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:28.949872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:28.949872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:28.949872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:28.949872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:28.949872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:28.949872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:28.949872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:28.949872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:31:28.949872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:28.949872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:31:28.949872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:31:28.950611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:28.950611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:31:28.950611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:28.950611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:31:28.950611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:31:28.950611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:28.950611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:31:28.950611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.950611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:31:28.951453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.951453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:28.951453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:28.951453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:28.951453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.951453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:31:28.951453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:31:28.951453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:31:28.951453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:28.951453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:28.951453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:28.951453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:28.952794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:28.952794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:28.952794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:28.952794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:28.952794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:28.952794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:28.952794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:28.952794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:28.952794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:28.952794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:28.952794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:28.952794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:28.952794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:28.952794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:28.954186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:28.954186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:28.954186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.954186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:28.954186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.954186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:28.954186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:28.954186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:28.954186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:28.954186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:28.954186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.954186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:28.954186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:28.954186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.954810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:28.954810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:28.954810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.954810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:28.954810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:28.954810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.954810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:31:28.954810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:31:28.954810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:31:28.954810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:28.954810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:28.954810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:28.954810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:28.954810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:28.954810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:28.954810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:28.954810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:28.954810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:28.954810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:28.955570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:28.955570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:28.955570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:28.955570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:28.955570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:28.955570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:28.955570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:28.955570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:28.955570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:28.955570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:28.955570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.955570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:28.955570 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:28.955570 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:28.957598 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:28.957598 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:28.957598 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:28.957598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.957598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:28.957598 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:28.957598 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:28.957598 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:28.957598 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:28.957598 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:28.957598 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.957598 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:28.959413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.959413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:31:28.959413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:28.959413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:28.959413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:28.959413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:28.959413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:28.959413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:28.959413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:28.959413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:28.959413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:28.959413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:31:28.959413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:28.959413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:31:28.959413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:31:28.959963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:28.959963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:31:28.959963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:28.959963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:31:28.959963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:31:28.959963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:28.959963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:31:28.959963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.959963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:31:28.959963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.959963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:28.959963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:28.959963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:28.959963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.960211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:31:28.960211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:31:28.960211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:28.960211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:28.960211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:28.960211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:28.960211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:28.960211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:28.960211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:28.960211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:28.960211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:28.960211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:28.960211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:28.960440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:28.960440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:28.960440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:28.960440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:28.960440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:28.960440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:28.960440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:28.960440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:28.960440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:28.960440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.960440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:28.960440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.960655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:28.960655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:28.960655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:28.960655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:28.960655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:28.960655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.960655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:28.960655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:28.960655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.960655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:28.960655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:28.960655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.960655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:28.960655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:28.960655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.960655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:31:28.960655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:31:28.960655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:28.960655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:28.961038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:28.961038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:28.961038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:28.961038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:28.961038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:28.961038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:28.961038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:28.961038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:28.961038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:28.961038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:28.961038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:28.961038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:28.961038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:28.961038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:28.961038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:28.961038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:28.961038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:28.961038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:28.961038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.961038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:28.962257 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:28.962546 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:28.962711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:28.962867 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:28.963339 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:28.963761 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:28.964119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.964896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:28.964896 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:28.964896 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:28.964896 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:28.964896 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:28.964896 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:28.964896 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.964896 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:28.965745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.965745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:28.965745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:28.965745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:28.965745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.965745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:31:28.965745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:28.965745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:28.965745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:28.965745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:28.965745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:28.965745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:28.965987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:28.965987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:28.965987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:28.965987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:28.965987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:28.965987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:28.965987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:28.965987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:28.965987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:28.965987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:28.965987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:28.965987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:28.965987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:28.966204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.966204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:28.966782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.966782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:28.966782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:28.966782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:28.966782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:28.966782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:28.966782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.966782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:28.966782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:28.966782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.966782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:28.966782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:28.966782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.966782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:28.966782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:28.966782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:28.967084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:31:28.967084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:28.967084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:28.967084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:28.967084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:28.967084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:28.967084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:28.967084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:28.967084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:28.967084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:28.967084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:28.967084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:28.967084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:28.967084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:28.967084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:28.967084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:28.967084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:28.967084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:28.967084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:28.967403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:28.967403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:28.967403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:33.659103 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:33.659744 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:33.659856 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:33.660708 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:33.702113 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:33.702640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.703692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:33.703692 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:33.703692 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:33.703692 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:33.703692 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:33.703692 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:33.703692 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.703692 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:33.705113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.705113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:33.705113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:33.705113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:33.705113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.705113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:31:33.705113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:33.705113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:33.705113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:33.705113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:33.705113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:33.705113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:33.705415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:33.705415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:33.705415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:33.705415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:33.705415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:33.705415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:33.705415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:33.705415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:33.705415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:33.705415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:33.705415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:33.705415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:33.705415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:33.705704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.705704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:33.706271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.706271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:33.706271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:33.706271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:33.706271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:33.706271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:33.706271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.706271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:33.706271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:33.706271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.706271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:33.706271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:33.706271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.706271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:33.706271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:33.706271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.706635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:31:33.706635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:33.706635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:33.706635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:33.706635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:33.706635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:33.706635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:33.706635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:33.706635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:33.706635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:33.706635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:33.706635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:33.706635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:33.706635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:33.706635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:33.706635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:33.706635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:33.706635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:33.706635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:33.707078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:33.707078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.707078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:33.893023 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:33.893438 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:33.893549 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:33.894328 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:33.894569 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:33.894767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.895290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:33.895290 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:33.895290 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:33.895290 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:33.895290 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:33.895290 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:33.895290 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.895290 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:33.896256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.896256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:33.896256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:33.896256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:33.896256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.896256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:31:33.896256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:33.896256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:33.896256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:33.896256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:33.896256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:33.896256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:33.896993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:33.896993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:33.896993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:33.896993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:33.896993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:33.896993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:33.896993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:33.896993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:33.896993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:33.896993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:33.896993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:33.896993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:33.896993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:33.897615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.897615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:33.897615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.897615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:33.897615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:33.897615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:33.897615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:33.897615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:33.897615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.897615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:33.897615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:33.897615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.897615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:33.897615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:33.897615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.898455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:33.898455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:33.898455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.898455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:31:33.898455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:33.898455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:33.898455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:33.898455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:33.898455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:33.898455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:33.898455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:33.898455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:33.898455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:33.898455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:33.898455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:33.898455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:33.898455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:33.898455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:33.898455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:33.899332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:33.899332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:33.899332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:33.899332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:33.899332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.899332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:33.946410 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:33.946864 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:33.947031 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:33.947224 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:31:33.948025 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:33.948332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.949336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:33.949336 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:33.949336 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:33.949336 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:33.949336 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:33.949336 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:33.949336 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.949336 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:33.950572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.950572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:31:33.950572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:33.950572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:33.950572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:33.950572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:33.950572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:33.950572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:33.950572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:33.950572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:33.950572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:33.950572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:31:33.950572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:33.950572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:31:33.950572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:31:33.951709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:33.951709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:31:33.951709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:33.951709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:31:33.951709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:31:33.951709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:33.951709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:31:33.951709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.951709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:31:33.952362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.952362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:33.952362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:33.952362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:33.952362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.952362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:31:33.952362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:31:33.952362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:31:33.952362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:33.952362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:33.952362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:33.952362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:33.953248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:33.953248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:33.953248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:33.953248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:33.953248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:33.953248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:33.953248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:33.953248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:33.953248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:33.953248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:33.953248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:33.953248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:33.953248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:33.953248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:33.954060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:33.954060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:33.954060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.954060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:33.954060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.954060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:33.954060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:33.954060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:33.954060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:33.954060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:33.954060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.954060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:33.954060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:33.954060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.954919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:33.954919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:33.954919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.954919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:33.954919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:33.954919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.954919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:31:33.954919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:31:33.954919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:31:33.954919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:33.954919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:33.954919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:33.954919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:33.954919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:33.954919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:33.954919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:33.954919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:33.954919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:33.954919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:33.956179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:33.956179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:33.956179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:33.956179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:33.956179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:33.956179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:33.956179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:33.956179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:33.956179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:33.956179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:33.956179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.956179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:33.956179 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:33.956179 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:33.957653 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:33.957653 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:33.957653 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:33.957653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.957653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:33.957653 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:33.957653 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:33.957653 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:33.957653 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:33.957653 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:33.957653 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.957653 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:33.958181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.958181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:31:33.958181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:33.958181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:33.958181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:33.958181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:33.958181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:33.958181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:33.958181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:33.958181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:33.958181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:33.958181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:31:33.958181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:33.958181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:31:33.958181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:31:33.958591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:33.958591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:31:33.958591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:33.958591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:31:33.958591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:31:33.958591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:33.958591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:31:33.958591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.958591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:31:33.958951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.958951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:33.958951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:33.958951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:33.958951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.958951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:31:33.958951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:31:33.958951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:33.958951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:33.958951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:33.958951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:33.959270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:33.959270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:33.959270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:33.959270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:33.959270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:33.959270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:33.959270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:33.959270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:33.959270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:33.959270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:33.959270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:33.959270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:33.959270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:33.959270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:33.959647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:33.959647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:33.959647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.959647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:33.960039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.960039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:33.960039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:33.960039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:33.960039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:33.960039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:33.960039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.960039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:33.960039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:33.960039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.960039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:33.960039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:33.960039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.960039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:33.960039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:33.960039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.960039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:31:33.960525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:31:33.960525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:33.960525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:33.960525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:33.960525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:33.960525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:33.960525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:33.960525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:33.960525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:33.960525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:33.960525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:33.960525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:33.960525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:33.960525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:33.960525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:33.960525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:33.960525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:33.960525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:33.960525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:33.961060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:33.961060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:33.961060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.961060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:33.962769 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:33.963055 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:33.963167 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:33.963328 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:33.964095 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:33.964546 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:33.964841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.965493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:33.965493 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:33.965493 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:33.965493 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:33.965493 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:33.965493 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:33.965493 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.965493 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:33.966404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.966404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:33.966404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:33.966404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:33.966404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.966404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:31:33.966404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:33.966404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:33.966404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:33.966404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:33.966404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:33.966404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:33.966945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:33.966945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:33.966945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:33.966945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:33.966945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:33.966945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:33.966945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:33.966945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:33.966945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:33.966945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:33.966945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:33.966945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:33.966945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:33.967424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.967424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:33.967903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.967903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:33.967903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:33.967903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:33.967903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:33.967903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:33.967903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.967903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:33.967903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:33.967903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.967903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:33.967903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:33.967903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.967903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:33.967903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:33.967903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:33.968520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:31:33.968520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:33.968520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:33.968520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:33.968520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:33.968520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:33.968520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:33.968520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:33.968520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:33.968520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:33.968520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:33.968520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:33.968520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:33.968520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:33.968520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:33.968520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:33.968520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:33.968520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:33.968520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:33.969338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:33.969338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:33.969338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:38.660051 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:38.661070 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:38.661070 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:38.662452 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:38.703063 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:38.703375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.704160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:38.704160 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:38.704160 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:38.704160 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:38.704160 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:38.704160 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:38.704160 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.704160 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:38.705089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.705089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:38.705089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:38.705089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:38.705089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.705089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:31:38.705089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:38.705089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:38.705089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:38.705089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:38.705089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:38.705089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:38.705715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:38.705715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:38.705715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:38.705715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:38.705715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:38.705715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:38.705715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:38.705715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:38.705715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:38.705715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:38.705715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:38.705715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:38.705715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:38.706277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.706277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:38.706277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.706277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:38.706277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:38.706277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:38.706277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:38.706277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:38.706277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.706277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:38.706277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:38.706277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.706277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:38.706277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:38.706277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.706944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:38.706944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:38.706944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.706944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:31:38.706944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:38.706944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:38.706944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:38.706944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:38.706944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:38.706944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:38.706944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:38.706944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:38.706944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:38.706944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:38.706944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:38.706944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:38.706944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:38.706944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:38.706944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:38.707825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:38.707825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:38.707825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:38.707825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:38.707825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.707825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:38.894527 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:38.895119 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:38.895241 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:38.896302 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:38.896606 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:38.896821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.897506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:38.897506 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:38.897506 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:38.897506 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:38.897506 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:38.897506 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:38.897506 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.897506 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:38.898596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.898596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:38.898596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:38.898596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:38.898596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.898596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:31:38.898596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:38.898596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:38.898596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:38.898596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:38.898596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:38.898596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:38.898975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:38.898975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:38.898975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:38.898975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:38.898975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:38.898975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:38.898975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:38.898975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:38.898975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:38.898975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:38.898975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:38.898975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:38.898975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:38.899316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.899316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:38.899768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.899768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:38.899768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:38.899768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:38.899768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:38.899768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:38.899768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.899768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:38.899768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:38.899768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.899768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:38.899768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:38.899768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.899768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:38.899768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:38.899768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.900225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:31:38.900225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:38.900225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:38.900225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:38.900225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:38.900225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:38.900225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:38.900225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:38.900225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:38.900225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:38.900225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:38.900225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:38.900225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:38.900225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:38.900225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:38.900225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:38.900225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:38.900225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:38.900225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:38.900718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:38.900718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.900718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:38.947037 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:38.948024 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:38.948285 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:38.948559 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:31:38.949391 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:38.949684 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.950504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:38.950504 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:38.950504 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:38.950504 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:38.950504 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:38.950504 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:38.950504 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.950504 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:38.952307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.952307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:31:38.952307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:38.952307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:38.952307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:38.952307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:38.952307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:38.952307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:38.952307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:38.952307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:38.952307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:38.952307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:31:38.952307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:38.952307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:31:38.952307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:31:38.952739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:38.952739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:31:38.952739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:38.952739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:31:38.952739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:31:38.952739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:38.952739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:31:38.952739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.952739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:31:38.953491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.953491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:38.953491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:38.953491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:38.953491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.953491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:31:38.953491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:31:38.953491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:31:38.953491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:38.953491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:38.953491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:38.953491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:38.953841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:38.953841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:38.953841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:38.953841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:38.953841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:38.953841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:38.953841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:38.953841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:38.953841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:38.953841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:38.953841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:38.953841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:38.953841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:38.953841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:38.954229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:38.954229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:38.954229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.954229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:38.954736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.954736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:38.954736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:38.954736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:38.954736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:38.954736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:38.954736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.954736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:38.954736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:38.954736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.954736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:38.954736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:38.954736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.954736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:38.954736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:38.954736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.955165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:31:38.955165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:31:38.955165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:31:38.955165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:38.955165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:38.955165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:38.955165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:38.955165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:38.955165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:38.955165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:38.955165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:38.955165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:38.955165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:38.955165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:38.955165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:38.955165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:38.955165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:38.955165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:38.955546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:38.955546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:38.955546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:38.955546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:38.955546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:38.955546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.955546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:38.955546 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:38.955736 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:38.955768 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:38.956026 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:38.956490 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:38.956789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.957404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:38.957404 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:38.957404 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:38.957404 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:38.957404 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:38.957404 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:38.957404 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.957404 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:38.958185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.958185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:31:38.958185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:38.958185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:38.958185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:38.958185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:38.958185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:38.958185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:38.958185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:38.958185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:38.958185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:38.958185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:31:38.958185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:38.958185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:31:38.958185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:31:38.958791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:38.958791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:31:38.958791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:38.958791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:31:38.958791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:31:38.958791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:38.958791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:31:38.958791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.958791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:31:38.959245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.959245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:38.959245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:38.959245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:38.959245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.959245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:31:38.959245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:31:38.959245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:38.959245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:38.959245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:38.959245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:38.959577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:38.959577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:38.959577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:38.959577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:38.959577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:38.959577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:38.959577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:38.959577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:38.959577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:38.959577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:38.959577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:38.959577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:38.959577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:38.959577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:38.960110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:38.960110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:38.960110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.960110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:38.960403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.960403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:38.960403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:38.960403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:38.960403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:38.960403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:38.960403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.960403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:38.960403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:38.960403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.960403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:38.960403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:38.960403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.960403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:38.960403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:38.960403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.960403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:31:38.961105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:31:38.961105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:38.961105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:38.961105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:38.961105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:38.961105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:38.961105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:38.961105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:38.961105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:38.961105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:38.961105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:38.961105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:38.961105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:38.961105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:38.961105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:38.961105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:38.961105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:38.961105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:38.961105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:38.961846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:38.961846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:38.961846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.961846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:38.963185 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:38.963524 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:38.963659 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:38.963953 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:38.964368 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:38.964648 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:38.964880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.965510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:38.965510 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:38.965510 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:38.965510 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:38.965510 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:38.965510 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:38.965510 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.965510 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:38.966578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.966578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:38.966578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:38.966578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:38.966578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.966578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:31:38.966578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:38.966578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:38.966578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:38.966578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:38.966578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:38.966578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:38.966943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:38.966943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:38.966943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:38.966943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:38.966943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:38.966943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:38.966943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:38.966943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:38.966943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:38.966943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:38.966943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:38.966943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:38.966943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:38.967291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.967291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:38.967688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.967688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:38.967688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:38.967688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:38.967688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:38.967688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:38.967688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.967688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:38.967688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:38.967688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.967688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:38.967688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:38.967688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.967688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:38.967688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:38.967688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:38.968270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:31:38.968270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:38.968270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:38.968270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:38.968270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:38.968270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:38.968270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:38.968270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:38.968270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:38.968270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:38.968270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:38.968270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:38.968270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:38.968270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:38.968270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:38.968270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:38.968270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:38.968270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:38.968270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:38.969167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:38.969167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:38.969167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:43.660527 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:43.661406 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:43.661406 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:43.662001 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:43.698710 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:43.698933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.699618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:43.699618 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:43.699618 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:43.699618 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:43.699618 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:43.699618 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:43.699618 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.699618 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:43.700651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.700651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:43.700651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:43.700651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:43.700651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.700651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:31:43.700651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:43.700651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:43.700651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:43.700651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:43.700651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:43.700651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:43.701427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:43.701427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:43.701427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:43.701427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:43.701427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:43.701427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:43.701427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:43.701427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:43.701427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:43.701427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:43.701427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:43.701427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:43.701427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:43.702134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.702134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:43.702134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.702134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:43.702134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:43.702134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:43.702134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:43.702134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:43.702134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.702134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:43.702134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:43.702134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.702134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:43.702134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:43.702134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.703016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:43.703016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:43.703016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.703016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:31:43.703016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:43.703016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:43.703016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:43.703016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:43.703016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:43.703016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:43.703016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:43.703016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:43.703016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:43.703016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:43.703016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:43.703016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:43.703016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:43.703016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:43.703016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:43.704163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:43.704163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:43.704163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:43.704163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:43.704163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.704163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:43.895209 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:43.895650 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:43.895834 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:43.896717 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:43.896980 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:43.897139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.897791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:43.897791 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:43.897791 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:43.897791 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:43.897791 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:43.897791 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:43.897791 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.897791 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:43.898897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.898897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:43.898897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:43.898897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:43.898897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.898897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:31:43.898897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:43.898897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:43.898897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:43.898897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:43.898897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:43.898897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:43.899520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:43.899520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:43.899520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:43.899520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:43.899520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:43.899520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:43.899520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:43.899520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:43.899520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:43.899520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:43.899520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:43.899520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:43.899520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:43.900161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.900161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:43.900161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.900161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:43.900161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:43.900161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:43.900161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:43.900161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:43.900161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.900161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:43.900161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:43.900161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.900161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:43.900161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:43.900161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.900913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:43.900913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:43.900913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.900913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:31:43.900913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:43.900913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:43.900913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:43.900913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:43.900913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:43.900913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:43.900913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:43.900913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:43.900913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:43.900913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:43.900913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:43.900913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:43.900913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:43.900913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:43.900913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:43.901811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:43.901811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:43.901811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:43.901811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:43.901811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.901811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:43.947932 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:43.948431 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:43.948609 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:43.948881 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:31:43.949585 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:43.949942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.950884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:43.950884 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:43.950884 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:43.950884 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:43.950884 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:43.950884 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:43.950884 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.950884 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:43.952321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.952321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:31:43.952321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:43.952321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:43.952321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:43.952321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:43.952321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:43.952321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:43.952321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:43.952321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:43.952321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:43.952321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:31:43.952321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:43.952321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:31:43.952321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:31:43.953328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:43.953328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:31:43.953328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:43.953328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:31:43.953328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:31:43.953328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:43.953328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:31:43.953328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.953328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:31:43.953899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.953899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:43.953899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:43.953899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:43.953899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.953899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:31:43.953899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:31:43.953899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:31:43.953899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:43.953899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:43.953899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:43.953899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:43.954419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:43.954419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:43.954419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:43.954419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:43.954419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:43.954419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:43.954419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:43.954419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:43.954419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:43.954419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:43.954419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:43.954419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:43.954419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:43.954419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:43.954815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:43.954815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:43.954815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.954815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:43.955547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.955547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:43.955547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:43.955547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:43.955547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:43.955547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:43.955547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.955547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:43.955547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:43.955547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.955547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:43.955547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:43.955547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.955547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:43.955547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:43.955547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.956020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:31:43.956020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:31:43.956020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:31:43.956020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:43.956020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:43.956020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:43.956020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:43.956020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:43.956020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:43.956020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:43.956020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:43.956020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:43.956020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:43.956020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:43.956020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:43.956020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:43.956020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:43.956020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:43.956497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:43.956497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:43.956497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:43.956497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:43.956497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:43.956497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.956497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:43.956497 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:43.956497 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:43.956497 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:43.956809 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:43.956954 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:43.957317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.958064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:43.958064 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:43.958064 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:43.958064 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:43.958064 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:43.958064 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:43.958064 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.958064 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:43.958643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.958643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:31:43.958643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:43.958643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:43.958643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:43.958643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:43.958643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:43.958643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:43.958643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:43.958643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:43.958643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:43.958643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:31:43.958643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:43.958643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:31:43.958643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:31:43.959536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:43.959536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:31:43.959536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:43.959536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:31:43.959536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:31:43.959536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:43.959536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:31:43.959536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.959536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:31:43.959536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.959536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:43.959536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:43.959536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:43.959536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.959999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:31:43.959999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:31:43.959999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:43.959999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:43.959999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:43.959999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:43.959999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:43.959999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:43.959999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:43.959999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:43.959999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:43.959999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:43.959999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:43.960484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:43.960484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:43.960484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:43.960484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:43.960484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:43.960484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:43.960484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:43.960484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:43.960484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:43.960484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.960484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:43.960484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.960872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:43.960872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:43.960872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:43.960872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:43.960872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:43.960872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.960872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:43.960872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:43.960872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.960872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:43.960872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:43.960872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.960872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:43.960872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:43.960872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.960872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:31:43.960872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:31:43.960872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:43.960872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:43.961391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:43.961391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:43.961391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:43.961391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:43.961391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:43.961391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:43.961391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:43.961391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:43.961391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:43.961391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:43.961391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:43.961391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:43.961391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:43.961391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:43.961391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:43.961391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:43.961391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:43.961391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:43.961391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.961391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:43.963555 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:43.963785 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:43.963862 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:43.963992 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:43.964413 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:43.964804 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:43.965130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.965886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:43.965886 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:43.965886 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:43.965886 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:43.965886 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:43.965886 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:43.965886 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.965886 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:43.966853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.966853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:43.966853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:43.966853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:43.966853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.966853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:31:43.966853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:43.966853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:43.966853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:43.966853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:43.966853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:43.966853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:43.967203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:43.967203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:43.967203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:43.967203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:43.967203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:43.967203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:43.967203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:43.967203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:43.967203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:43.967203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:43.967203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:43.967203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:43.967203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:43.967518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.967518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:43.968061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.968061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:43.968061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:43.968061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:43.968061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:43.968061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:43.968061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.968061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:43.968061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:43.968061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.968061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:43.968061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:43.968061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.968061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:43.968061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:43.968061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:43.968759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:31:43.968759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:43.968759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:43.968759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:43.968759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:43.968759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:43.968759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:43.968759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:43.968759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:43.968759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:43.968759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:43.968759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:43.968759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:43.968759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:43.968759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:43.968759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:43.968759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:43.968759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:43.968759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:43.969385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:43.969385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:43.969385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:48.661264 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:48.662200 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:48.662326 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:48.663211 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:48.701617 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:48.701617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.702137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:48.702137 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:48.702137 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:48.702137 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:48.702137 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:48.702137 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:48.702137 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.702137 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:48.702921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.702921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:48.702921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:48.702921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:48.702921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.702921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:31:48.702921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:48.702921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:48.702921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:48.702921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:48.702921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:48.702921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:48.703600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:48.703600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:48.703600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:48.703600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:48.703600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:48.703600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:48.703600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:48.703600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:48.703600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:48.703600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:48.703600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:48.703600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:48.703600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:48.704210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.704210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:48.704210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.704210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:48.704210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:48.704210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:48.704210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:48.704210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:48.704210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.704210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:48.704210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:48.704210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.704210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:48.704210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:48.704210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.705025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:48.705025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:48.705025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.705025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:31:48.705025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:48.705025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:48.705025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:48.705025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:48.705025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:48.705025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:48.705025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:48.705025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:48.705025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:48.705025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:48.705025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:48.705025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:48.705025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:48.705025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:48.705025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:48.705661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:48.705661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:48.705661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:48.705661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:48.705661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.705661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:48.895726 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:48.896083 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:48.896177 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:48.897004 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:48.897323 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:48.897666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.898825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:48.898825 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:48.898825 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:48.898825 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:48.898825 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:48.898825 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:48.898825 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.898825 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:48.899803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.899803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:48.899803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:48.899803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:48.899803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.899803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:31:48.899803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:48.899803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:48.899803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:48.899803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:48.899803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:48.899803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:48.900582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:48.900582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:48.900582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:48.900582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:48.900582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:48.900582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:48.900582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:48.900582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:48.900582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:48.900582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:48.900582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:48.900582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:48.900582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:48.901431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.901431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:48.901431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.901431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:48.901431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:48.901431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:48.901431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:48.901431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:48.901431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.901431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:48.901431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:48.901431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.901431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:48.901431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:48.901431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.902417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:48.902417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:48.902417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.902417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:31:48.902417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:48.902417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:48.902417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:48.902417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:48.902417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:48.902417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:48.902417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:48.902417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:48.902417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:48.902417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:48.902417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:48.902417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:48.902417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:48.902417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:48.902417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:48.903914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:48.903914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:48.903914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:48.903914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:48.903914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.903914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:48.948528 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:48.948965 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:48.949174 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:48.949379 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:31:48.949954 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:48.950172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.950899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:48.950899 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:48.950899 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:48.950899 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:48.950899 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:48.950899 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:48.950899 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.950899 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:48.952136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.952136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:31:48.952136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:48.952136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:48.952136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:48.952136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:48.952136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:48.952136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:48.952136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:48.952136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:48.952136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:48.952136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:31:48.952136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:48.952136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:31:48.952136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:31:48.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:48.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:31:48.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:48.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:31:48.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:31:48.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:48.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:31:48.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.952806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:31:48.953463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.953463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:48.953463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:48.953463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:48.953463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.953463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:31:48.953463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:31:48.953463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:31:48.953463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:48.953463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:48.953463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:48.953463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:48.954017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:48.954017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:48.954017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:48.954017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:48.954017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:48.954017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:48.954017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:48.954017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:48.954017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:48.954017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:48.954017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:48.954017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:48.954017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:48.954017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:48.954587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:48.954587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:48.954587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.954587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:48.955422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.955422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:48.955422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:48.955422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:48.955422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:48.955422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:48.955422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.955422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:48.955422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:48.955422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.955422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:48.955422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:48.955422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.955422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:48.955422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:48.955422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.956110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:31:48.956110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:31:48.956110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:31:48.956110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:48.956110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:48.956110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:48.956110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:48.956110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:48.956110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:48.956110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:48.956110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:48.956110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:48.956110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:48.956110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:48.956110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:48.956110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:48.956110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:48.956110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:48.956792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:48.956792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:48.956792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:48.956792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:48.956792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:48.956792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.956792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:48.956792 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:48.957168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:48.957168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:48.957259 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:48.957627 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:48.957933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.958530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:48.958530 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:48.958530 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:48.958530 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:48.958530 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:48.958530 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:48.958530 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.958530 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:48.959704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.959704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:31:48.959704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:48.959704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:48.959704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:48.959704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:48.959704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:48.959704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:48.959704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:48.959704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:48.959704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:48.959704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:31:48.959704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:48.959704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:31:48.959704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:31:48.960140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:48.960140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:31:48.960140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:48.960140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:31:48.960140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:31:48.960140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:48.960140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:31:48.960140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.960140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:31:48.961257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.961257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:48.961257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:48.961257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:48.961257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.961257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:31:48.961257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:31:48.961257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:48.961257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:48.961257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:48.961257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:48.961608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:48.961608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:48.961608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:48.961608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:48.961608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:48.961608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:48.961608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:48.961608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:48.961608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:48.961608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:48.961608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:48.961608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:48.961608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:48.961608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:48.962175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:48.962175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:48.962175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.962175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:48.962645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.962645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:48.962645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:48.962645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:48.962645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:48.962645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:48.962645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.962645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:48.962645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:48.962645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.962645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:48.962645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:48.962645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.962645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:48.962645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:48.962645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.962645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:31:48.963318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:31:48.963318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:48.963318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:48.963318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:48.963318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:48.963318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:48.963318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:48.963318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:48.963318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:48.963318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:48.963318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:48.963318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:48.963318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:48.963318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:48.963318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:48.963318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:48.963318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:48.963318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:48.963318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:48.963902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:48.963902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:48.963902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.963902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:48.964080 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:48.964412 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:48.964514 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:48.964709 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:48.965064 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:48.965328 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:48.965536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.966180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:48.966180 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:48.966180 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:48.966180 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:48.966180 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:48.966180 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:48.966180 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.966180 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:48.967134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.967134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:48.967134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:48.967134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:48.967134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.967134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:31:48.967134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:48.967134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:48.967134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:48.967134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:48.967134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:48.967134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:48.967928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:48.967928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:48.967928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:48.967928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:48.967928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:48.967928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:48.967928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:48.967928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:48.967928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:48.967928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:48.967928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:48.967928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:48.967928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:48.968791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.968791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:48.968791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.968791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:48.968791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:48.968791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:48.968791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:48.968791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:48.968791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.968791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:48.968791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:48.968791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.968791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:48.968791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:48.968791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:48.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:48.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:48.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:31:48.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:48.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:48.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:48.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:48.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:48.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:48.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:48.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:48.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:48.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:48.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:48.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:48.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:48.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:48.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:48.970382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:48.970382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:48.970382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:48.970382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:48.970382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:48.970382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:53.662349 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:53.663036 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:53.663036 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:53.663829 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:53.694754 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:53.694993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.695572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:53.695572 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:53.695572 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:53.695572 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:53.695572 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:53.695572 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:53.695572 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.695572 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:53.696246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.696246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:53.696246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:53.696246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:53.696246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.696246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:31:53.696246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:53.696246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:53.696246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:53.696246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:53.696246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:53.696246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:53.696742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:53.696742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:53.696742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:53.696742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:53.696742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:53.696742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:53.696742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:53.696742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:53.696742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:53.696742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:53.696742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:53.696742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:53.696742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:53.697391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.697391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:53.697391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.697391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:53.697391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:53.697391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:53.697391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:53.697391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:53.697391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.697391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:53.697391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:53.697391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.697391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:53.697391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:53.697391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.698171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:53.698171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:53.698171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.698171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:31:53.698171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:53.698171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:53.698171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:53.698171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:53.698171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:53.698171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:53.698171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:53.698171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:53.698171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:53.698171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:53.698171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:53.698171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:53.698171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:53.698171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:53.698171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:53.699110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:53.699110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:53.699110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:53.699110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:53.699110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.699110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:53.896207 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:53.896705 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:53.896825 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:53.897502 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:53.897690 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:53.897882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.898453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:53.898453 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:53.898453 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:53.898453 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:53.898453 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:53.898453 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:53.898453 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.898453 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:53.899717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.899717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:53.899717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:53.899717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:53.899717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.899717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:31:53.899717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:53.899717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:53.899717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:53.899717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:53.899717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:53.899717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:53.900270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:53.900270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:53.900270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:53.900270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:53.900270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:53.900270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:53.900270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:53.900270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:53.900270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:53.900270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:53.900270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:53.900270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:53.900270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:53.900866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.900866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:53.900866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.900866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:53.900866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:53.900866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:53.900866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:53.900866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:53.900866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.900866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:53.900866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:53.900866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.900866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:53.900866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:53.900866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.901531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:53.901531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:53.901531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.901531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:31:53.901531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:53.901531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:53.901531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:53.901531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:53.901531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:53.901531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:53.901531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:53.901531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:53.901531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:53.901531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:53.901531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:53.901531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:53.901531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:53.901531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:53.901531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:53.902351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:53.902351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:53.902351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:53.902351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:53.902351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.902351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:53.949097 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:53.949653 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:53.949868 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:53.950061 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:31:53.950643 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:53.950992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.951799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:53.951799 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:53.951799 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:53.951799 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:53.951799 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:53.951799 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:53.951799 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.951799 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:53.952838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.952838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:31:53.952838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:53.952838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:53.952838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:53.952838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:53.952838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:53.952838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:53.952838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:53.952838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:53.952838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:53.952838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:31:53.952838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:53.952838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:31:53.952838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:31:53.953780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:53.953780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:31:53.953780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:53.953780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:31:53.953780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:31:53.953780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:53.953780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:31:53.953780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.953780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:31:53.953780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.953780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:53.953780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:53.953780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:53.953780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.954494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:31:53.954494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:31:53.954494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:31:53.954494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:53.954494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:53.954494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:53.954494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:53.954494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:53.954494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:53.954494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:53.954494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:53.954494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:53.954494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:53.955245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:53.955245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:53.955245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:53.955245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:53.955245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:53.955245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:53.955245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:53.955245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:53.955245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:53.955245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:53.955245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.955245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:53.955849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.955849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:53.955849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:53.955849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:53.955849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:53.955849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:53.955849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.955849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:53.955849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:53.955849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.955849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:53.955849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:53.955849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.955849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:53.955849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:53.955849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.956540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:31:53.956540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:31:53.956540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:31:53.956540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:53.956540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:53.956540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:53.956540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:53.956540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:53.956540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:53.956540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:53.956540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:53.956540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:53.956540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:53.956540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:53.956540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:53.956540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:53.956540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:53.956540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:53.957114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:53.957114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:53.957114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:53.957114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:53.957114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:53.957114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.957114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:53.957114 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:53.957114 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:53.957389 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:53.957389 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:53.957697 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:53.958014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.958752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:53.958752 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:53.958752 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:53.958752 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:53.958752 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:53.958752 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:53.958752 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.958752 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:53.960100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.960100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:31:53.960100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:53.960100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:53.960100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:53.960100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:53.960100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:53.960100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:53.960100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:53.960100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:53.960100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:53.960100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:31:53.960100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:53.960100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:31:53.960100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:31:53.960510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:53.960510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:31:53.960510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:53.960510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:31:53.960510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:31:53.960510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:53.960510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:31:53.960510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.960510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:31:53.961630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.961630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:53.961630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:53.961630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:53.961630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.961630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:31:53.961630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:31:53.961630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:53.961630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:53.961630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:53.961630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:53.961987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:53.961987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:53.961987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:53.961987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:53.961987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:53.961987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:53.961987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:53.961987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:53.961987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:53.961987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:53.961987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:53.961987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:53.961987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:53.961987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:53.962376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:53.962376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:53.962376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.962376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:53.963874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.963874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:53.963874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:53.963874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:53.963874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:53.963874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:53.963874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.963874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:53.963874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:53.963874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.963874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:53.963874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:53.963874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.963874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:53.963874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:53.963874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.963874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:31:53.964370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:31:53.964370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:53.964370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:53.964370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:53.964370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:53.964370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:53.964370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:53.964370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:53.964370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:53.964370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:53.964370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:53.964370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:53.964370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:53.964370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:53.964370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:53.964370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:53.964370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:53.964370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:53.964370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:53.964922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:53.964922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:53.964922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.964922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:53.964922 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:53.964922 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:53.964922 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:53.965149 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:53.966233 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:53.966504 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:53.966652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.967100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:53.967100 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:53.967100 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:53.967100 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:53.967100 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:53.967100 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:53.967100 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.967100 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:53.967955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.967955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:53.967955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:53.967955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:53.967955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.967955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:31:53.967955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:53.967955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:53.967955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:53.967955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:53.967955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:53.967955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:53.968432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:53.968432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:53.968432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:53.968432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:53.968432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:53.968432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:53.968432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:53.968432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:53.968432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:53.968432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:53.968432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:53.968432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:53.968432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:53.968821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.968821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:53.968965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.968965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:53.968965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:53.968965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:53.968965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:53.968965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:53.968965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.968965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:53.968965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:53.968965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.968965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:53.968965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:53.968965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.968965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:53.968965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:53.968965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:53.969429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:31:53.969429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:53.969429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:53.969429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:53.969429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:53.969429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:53.969429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:53.969429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:53.969429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:53.969429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:53.969429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:53.969429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:53.969429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:53.969429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:53.969429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:53.969429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:53.969429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:53.969429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:53.969429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:53.970003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:53.970003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:53.970003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:58.663023 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:58.663765 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:58.664028 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:58.665332 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:58.705852 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:58.706275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.707196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:58.707196 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:58.707196 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:58.707196 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:58.707196 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:58.707196 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:58.707196 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.707196 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:58.708189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.708189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:58.708189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:58.708189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:58.708189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.708189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:31:58.708189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:58.708189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:58.708189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:58.708189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:58.708189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:58.708189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:58.708767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:58.708767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:58.708767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:58.708767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:58.708767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:58.708767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:58.708767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:58.708767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:58.708767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:58.708767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:58.708767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:58.708767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:58.708767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:58.709282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.709282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:58.709282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.709282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:58.709282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:58.709282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:58.709282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:58.709282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:58.709282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.709282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:58.709282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:58.709282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.709282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:58.709282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:58.709282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.709920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:58.709920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:58.709920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.709920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:31:58.709920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:58.709920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:58.709920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:58.709920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:58.709920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:58.709920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:58.709920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:58.709920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:58.709920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:58.709920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:58.709920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:58.709920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:58.709920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:58.709920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:58.709920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:58.710770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:58.710770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:58.710770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:58.710770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:58.710770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.710770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:58.898079 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:58.898860 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:58.899065 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:58.901476 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:58.901476 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:58.901704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.903118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:58.903118 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:58.903118 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:58.903118 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:58.903118 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:58.903118 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:58.903118 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.903118 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:58.904458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.904458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:58.904458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:58.904458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:58.904458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.904458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:31:58.904458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:58.904458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:58.904458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:58.904458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:58.904458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:58.904458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:58.905192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:58.905192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:58.905192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:58.905192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:58.905192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:58.905192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:58.905192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:58.905192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:58.905192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:58.905192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:58.905192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:58.905192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:58.905192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:58.905853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.905853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:58.905853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.905853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:58.905853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:58.905853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:58.905853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:58.905853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:58.905853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.905853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:58.905853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:58.905853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.905853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:58.905853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:58.905853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.906633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:58.906633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:58.906633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.906633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:31:58.906633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:58.906633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:58.906633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:58.906633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:58.906633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:58.906633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:58.906633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:58.906633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:58.906633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:58.906633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:58.906633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:58.906633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:58.906633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:58.906633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:58.906633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:58.907552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:58.907552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:58.907552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:58.907552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:58.907552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.907552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:58.949823 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:58.950279 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:58.950469 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:58.950643 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:31:58.951709 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:58.952283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.953575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:58.953575 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:58.953575 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:58.953575 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:58.953575 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:58.953575 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:58.953575 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.953575 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:58.954840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.954840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:31:58.954840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:58.954840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:58.954840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:58.954840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:58.954840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:58.954840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:58.954840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:58.954840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:58.954840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:58.954840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:31:58.954840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:58.954840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:31:58.954840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:31:58.955582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:58.955582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:31:58.955582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:58.955582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:31:58.955582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:31:58.955582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:58.955582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:31:58.955582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.955582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:31:58.956003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.956003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:58.956003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:58.956003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:58.956003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.956003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:31:58.956003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:31:58.956003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:31:58.956003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:58.956003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:58.956003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:58.956003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:58.956529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:58.956529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:58.956529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:58.956529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:58.956529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:58.956529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:58.956529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:58.956529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:58.956529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:58.956529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:58.956529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:58.956529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:58.956529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:58.956529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:58.957153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:58.957153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:58.957153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.957153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:58.957153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.957153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:58.957153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:58.957153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:58.957153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:58.957153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:58.957153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.957153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:58.957153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:58.957153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.957778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:58.957778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:58.957778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.957778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:58.957778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:58.957778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.957778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:31:58.957778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:31:58.957778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:31:58.957778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:58.957778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:58.957778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:58.957778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:58.957778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:58.957778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:58.957778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:58.957778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:58.957778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:58.957778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:58.958661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:58.958661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:58.958661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:58.958661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:58.958661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:58.958661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:58.958661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:58.958661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:58.958661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:58.958661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:58.958661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.958661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:58.958661 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:58.958661 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:58.959305 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:58.959305 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:58.959305 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:58.959305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.959502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:58.959502 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:58.959502 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:58.959502 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:58.959502 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:58.959502 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:58.959502 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.959502 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:58.960175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.960175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:31:58.960175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:58.960175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:58.960175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:58.960175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:58.960175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:58.960175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:58.960175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:58.960175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:31:58.960175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:58.960175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:31:58.960175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:58.960175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:31:58.960175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:31:58.960876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:58.960876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:31:58.960876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:58.960876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:31:58.960876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:31:58.960876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:58.960876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:31:58.960876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.960876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:31:58.961276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.961276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:58.961276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:58.961276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:58.961276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.961276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:31:58.961276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:31:58.961276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:58.961276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:58.961276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:58.961276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:58.961766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:58.961766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:58.961766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:58.961766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:58.961766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:58.961766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:58.961766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:58.961766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:58.961766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:58.961766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:58.961766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:58.961766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:58.961766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:58.961766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:58.962524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:58.962524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:58.962524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.962524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:58.962524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.962524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:58.962524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:58.962524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:58.962524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:58.962524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:58.962524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.962524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:58.962524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:58.962524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:58.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:58.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:58.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:58.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:31:58.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:31:58.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:31:58.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:31:58.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:58.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:58.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:58.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:58.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:58.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:58.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:58.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:58.962914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:58.963281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:58.963281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:58.963281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:58.963281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:58.963281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:58.963281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:58.963281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:58.963281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:58.963281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:58.963281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.963281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:31:58.964801 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:31:58.965075 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:31:58.965208 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:31:58.965276 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:31:58.965801 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:58.966093 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:31:58.966335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.966948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:31:58.966948 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:58.966948 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:31:58.966948 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:31:58.966948 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:58.966948 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:31:58.966948 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.966948 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:31:58.967885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.967885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:31:58.967885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:58.967885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:31:58.967885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.967885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:31:58.967885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:58.967885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:58.967885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:58.967885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:58.967885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:58.967885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:58.968471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:31:58.968471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:58.968471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:31:58.968471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:58.968471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:31:58.968471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:31:58.968471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:58.968471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:31:58.968471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:58.968471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:31:58.968471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:31:58.968471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:58.968471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:31:58.969061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.969061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:31:58.969061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.969061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:31:58.969061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:31:58.969061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:31:58.969061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:31:58.969061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:31:58.969061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.969061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:31:58.969061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:31:58.969061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.969061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:31:58.969061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:31:58.969061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.969725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:31:58.969725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:31:58.969725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:31:58.969725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:31:58.969725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:31:58.969725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:58.969725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:58.969725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:31:58.969725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:31:58.969725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:31:58.969725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:31:58.969725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:31:58.969725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:31:58.969725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:31:58.969725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:31:58.969725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:31:58.969725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:31:58.969725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:31:58.969725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:31:58.970570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:31:58.970570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:31:58.970570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:31:58.970570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:31:58.970570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:31:58.970570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:03.663665 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:03.664370 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:03.664445 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:03.665727 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:03.699515 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:03.699846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.702111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:03.702111 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:03.702111 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:03.702111 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:03.702111 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:03.702111 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:03.702111 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.702111 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:03.704371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.704371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:03.704371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:03.704371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:03.704371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.704371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:32:03.704371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:03.704371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:03.704371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:03.704371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:03.704371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:03.704371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:03.705172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:03.705172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:03.705172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:03.705172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:03.705172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:03.705172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:03.705172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:03.705172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:03.705172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:03.705172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:03.705172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:03.705172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:03.705172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:03.706080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.706080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:03.706080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.706080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:03.706080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:03.706080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:03.706080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:03.706080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:03.706080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.706080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:03.706080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:03.706080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.706080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:03.706080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:03.706080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.706927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:03.706927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:03.706927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.706927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:32:03.706927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:03.706927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:03.706927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:03.706927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:03.706927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:03.706927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:03.706927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:03.706927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:03.706927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:03.706927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:03.706927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:03.706927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:03.706927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:03.706927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:03.706927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:03.708130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:03.708130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:03.708130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:03.708130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:03.708130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.708130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:03.898862 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:03.899335 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:03.899540 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:03.900551 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:03.900822 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:03.900985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.901721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:03.901721 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:03.901721 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:03.901721 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:03.901721 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:03.901721 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:03.901721 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.901721 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:03.903240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.903240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:03.903240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:03.903240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:03.903240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.903240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:32:03.903240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:03.903240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:03.903240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:03.903240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:03.903240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:03.903240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:03.903598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:03.903598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:03.903598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:03.903598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:03.903598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:03.903598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:03.903598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:03.903598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:03.903598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:03.903598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:03.903598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:03.903598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:03.903598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:03.903981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.903981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:03.905072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.905072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:03.905072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:03.905072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:03.905072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:03.905072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:03.905072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.905072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:03.905072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:03.905072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.905072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:03.905072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:03.905072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.905072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:03.905072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:03.905072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.905519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:32:03.905519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:03.905519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:03.905519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:03.905519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:03.905519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:03.905519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:03.905519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:03.905519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:03.905519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:03.905519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:03.905519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:03.905519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:03.905519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:03.905519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:03.905519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:03.905519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:03.905519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:03.905519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:03.906048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:03.906048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.906048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:03.950465 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:03.950978 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:03.951177 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:03.951372 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:32:03.952059 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:03.952426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.953398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:03.953398 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:03.953398 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:03.953398 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:03.953398 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:03.953398 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:03.953398 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.953398 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:03.954639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.954639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:32:03.954639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:03.954639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:03.954639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:03.954639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:03.954639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:03.954639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:03.954639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:03.954639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:03.954639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:03.954639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:32:03.954639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:03.954639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:32:03.954639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:32:03.955420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:03.955420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:32:03.955420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:03.955420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:32:03.955420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:32:03.955420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:03.955420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:32:03.955420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.955420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:32:03.955989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.955989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:03.955989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:03.955989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:03.955989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.955989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:32:03.955989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:32:03.955989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:32:03.955989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:03.955989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:03.955989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:03.955989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:03.956564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:03.956564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:03.956564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:03.956564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:03.956564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:03.956564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:03.956564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:03.956564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:03.956564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:03.956564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:03.956564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:03.956564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:03.956564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:03.956564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:03.957245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:03.957245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:03.957245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.957245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:03.957650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.957650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:03.957650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:03.957650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:03.957650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:03.957650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:03.957650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.957650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:03.957650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:03.957650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.957650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:03.957650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:03.957650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.957650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:03.957650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:03.957650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.958399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:32:03.958399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:32:03.958399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:32:03.958399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:03.958399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:03.958399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:03.958399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:03.958399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:03.958399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:03.958399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:03.958399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:03.958399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:03.958399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:03.958399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:03.958399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:03.958399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:03.958399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:03.958399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:03.959293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:03.959293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:03.959293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:03.959293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:03.959293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:03.959293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.959293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:03.959293 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:03.959293 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:03.959293 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:03.959830 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:03.959830 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:03.959830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.960345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:03.960345 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:03.960345 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:03.960345 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:03.960345 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:03.960345 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:03.960345 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.960345 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:03.960993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.960993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:32:03.960993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:03.960993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:03.960993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:03.960993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:03.960993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:03.960993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:03.960993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:03.960993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:03.960993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:03.960993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:32:03.960993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:03.960993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:32:03.960993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:32:03.961410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:03.961410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:32:03.961410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:03.961410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:32:03.961410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:32:03.961410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:03.961410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:32:03.961410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.961410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:32:03.961655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.961655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:03.961655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:03.961655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:03.961655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.961655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:32:03.961655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:32:03.961655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:03.961655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:03.961655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:03.961655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:03.962015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:03.962015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:03.962015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:03.962015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:03.962015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:03.962015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:03.962015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:03.962015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:03.962015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:03.962015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:03.962015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:03.962015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:03.962015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:03.962015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:03.962411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:03.962411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:03.962411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.962411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:03.962411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.962411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:03.962411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:03.962411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:03.962411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:03.962411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:03.962411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.962411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:03.962411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:03.962411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.962832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:03.962832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:03.962832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.962832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:03.962832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:03.962832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.962832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:32:03.962832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:32:03.962832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:03.962832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:03.962832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:03.962832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:03.962832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:03.962832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:03.962832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:03.962832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:03.962832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:03.962832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:03.962832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:03.963351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:03.963351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:03.963351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:03.963351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:03.963351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:03.963351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:03.963351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:03.963351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:03.963351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:03.963351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.963351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:03.965195 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:03.965473 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:03.965551 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:03.965698 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:03.966115 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:03.966467 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:03.966733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.967450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:03.967450 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:03.967450 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:03.967450 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:03.967450 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:03.967450 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:03.967450 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.967450 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:03.968209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.968209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:03.968209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:03.968209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:03.968209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.968209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:32:03.968209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:03.968209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:03.968209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:03.968209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:03.968209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:03.968209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:03.968572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:03.968572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:03.968572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:03.968572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:03.968572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:03.968572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:03.968572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:03.968572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:03.968572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:03.968572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:03.968572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:03.968572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:03.968572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:03.968995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.968995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:03.969250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.969250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:03.969250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:03.969250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:03.969250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:03.969250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:03.969250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.969250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:03.969250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:03.969250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.969250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:03.969250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:03.969250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.969250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:03.969250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:03.969250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:03.969730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:32:03.969730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:03.969730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:03.969730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:03.969730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:03.969730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:03.969730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:03.969730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:03.969730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:03.969730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:03.969730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:03.969730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:03.969730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:03.969730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:03.969730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:03.969730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:03.969730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:03.969730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:03.969730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:03.970271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:03.970271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:03.970271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:08.664409 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:08.665073 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:08.665073 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:08.665910 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:08.668066 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=454 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:32:08.704651 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:08.704651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.704651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:08.704651 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:08.704651 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:08.704651 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:08.704651 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:08.704651 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:08.704651 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.704651 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:08.705392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.705392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:08.705392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:08.705392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:08.705392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.705392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:32:08.705392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:08.705392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:08.705392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:08.705392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:08.705392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:08.705392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:08.705805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:08.705805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:08.705805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:08.705805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:08.705805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:08.705805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:08.705805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:08.705805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:08.705805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:08.705805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:08.705805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:08.705805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:08.705805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:08.706230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.706230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:08.706230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.706230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:08.706230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:08.706230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:08.706230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:08.706230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:08.706230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.706230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:08.706230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:08.706230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.706230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:08.706230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:08.706230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.706776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:08.706776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:08.706776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.706776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:32:08.706776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:08.706776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:08.706776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:08.706776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:08.706776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:08.706776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:08.706776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:08.706776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:08.706776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:08.706776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:08.706776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:08.706776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:08.706776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:08.706776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:08.706776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:08.707363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:08.707363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:08.707363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:08.707363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:08.707363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.707363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:08.899842 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:08.900370 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:08.900553 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:08.902182 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:08.902182 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:08.902182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.902864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:08.902864 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:08.902864 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:08.902864 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:08.902864 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:08.902864 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:08.902864 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.902864 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:08.903912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.903912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:08.903912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:08.903912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:08.903912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.903912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:32:08.903912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:08.903912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:08.903912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:08.903912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:08.903912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:08.903912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:08.904246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:08.904246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:08.904246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:08.904246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:08.904246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:08.904246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:08.904246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:08.904246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:08.904246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:08.904246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:08.904246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:08.904246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:08.904246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:08.904630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.904630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:08.905530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.905530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:08.905530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:08.905530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:08.905530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:08.905530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:08.905530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.905530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:08.905530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:08.905530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.905530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:08.905530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:08.905530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.905530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:08.905530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:08.905530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.906052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:32:08.906052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:08.906052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:08.906052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:08.906052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:08.906052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:08.906052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:08.906052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:08.906052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:08.906052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:08.906052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:08.906052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:08.906052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:08.906052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:08.906052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:08.906052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:08.906052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:08.906052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:08.906052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:08.906593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:08.906593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.906593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:08.950986 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:08.951274 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:08.951363 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:08.951458 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:32:08.952815 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:08.952815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.953212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:08.953212 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:08.953212 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:08.953212 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:08.953212 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:08.953212 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:08.953212 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.953212 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:08.953963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.953963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:32:08.953963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:08.953963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:08.953963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:08.953963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:08.953963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:08.953963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:08.953963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:08.953963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:08.953963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:08.953963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:32:08.953963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:08.953963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:32:08.953963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:32:08.954450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:08.954450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:32:08.954450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:08.954450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:32:08.954450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:32:08.954450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:08.954450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:32:08.954450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.954450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:32:08.954796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.954796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:08.954796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:08.954796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:08.954796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.954796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:32:08.954796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:32:08.954796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:32:08.954796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:08.954796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:08.954796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:08.954796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:08.955151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:08.955151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:08.955151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:08.955151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:08.955151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:08.955151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:08.955151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:08.955151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:08.955151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:08.955151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:08.955151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:08.955151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:08.955151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:08.955151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:08.955506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:08.955506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:08.955506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.955506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:08.955796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.955796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:08.955796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:08.955796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:08.955796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:08.955796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:08.955796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.955796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:08.955796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:08.955796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.955796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:08.955796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:08.955796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.955796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:08.955796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:08.955796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.956628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:32:08.956628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:32:08.956628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:32:08.956628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:08.956628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:08.956628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:08.956628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:08.956628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:08.956628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:08.956628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:08.956628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:08.956628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:08.956628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:08.956628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:08.956628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:08.956628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:08.956628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:08.956628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:08.957629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:08.957629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:08.957629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:08.957629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:08.957629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:08.957629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.957629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:08.958301 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:08.958550 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:08.958632 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:08.958766 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:08.959126 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:08.959405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.959894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:08.959894 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:08.959894 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:08.959894 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:08.959894 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:08.959894 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:08.959894 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.959894 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:08.960571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.960571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:32:08.960571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:08.960571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:08.960571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:08.960571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:08.960571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:08.960571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:08.960571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:08.960571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:08.960571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:08.960571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:32:08.960571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:08.960571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:32:08.960571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:32:08.961178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:08.961178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:32:08.961178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:08.961178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:32:08.961178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:32:08.961178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:08.961178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:32:08.961178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.961178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:32:08.961505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.961505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:08.961505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:08.961505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:08.961505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.961505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:32:08.961505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:32:08.961505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:08.961505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:08.961505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:08.961505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:08.961998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:08.961998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:08.961998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:08.961998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:08.961998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:08.961998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:08.961998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:08.961998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:08.961998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:08.961998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:08.961998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:08.961998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:08.961998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:08.961998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:08.962401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:08.962401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:08.962401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.962401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:08.962401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.962401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:08.962401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:08.962401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:08.962401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:08.962401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:08.962401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.962401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:08.962401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:08.962401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.962815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:08.962815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:08.962815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.962815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:08.962815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:08.962815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.962815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:32:08.962815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:32:08.962815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:08.962815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:08.962815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:08.962815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:08.962815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:08.962815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:08.962815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:08.962815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:08.962815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:08.962815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:08.962815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:08.963444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:08.963444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:08.963444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:08.963444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:08.963444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:08.963444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:08.963444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:08.963444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:08.963444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:08.963444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.963444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:08.965618 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:08.965928 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:08.966005 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:08.966145 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:08.966625 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:08.967063 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:08.967310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.968096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:08.968096 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:08.968096 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:08.968096 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:08.968096 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:08.968096 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:08.968096 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.968096 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:08.968901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.968901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:08.968901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:08.968901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:08.968901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.968901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:32:08.968901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:08.968901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:08.968901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:08.968901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:08.968901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:08.968901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:08.969239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:08.969239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:08.969239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:08.969239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:08.969239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:08.969239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:08.969239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:08.969239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:08.969239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:08.969239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:08.969239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:08.969239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:08.969239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:08.969662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.969662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:08.969809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.969809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:08.969809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:08.969809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:08.969809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:08.969809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:08.969809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.969809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:08.969809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:08.969809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.969809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:08.969809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:08.969809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.969809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:08.969809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:08.969809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:08.970228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:32:08.970228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:08.970228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:08.970228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:08.970228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:08.970228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:08.970228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:08.970228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:08.970228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:08.970228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:08.970228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:08.970228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:08.970228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:08.970228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:08.970228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:08.970228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:08.970228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:08.970228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:08.970228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:08.970789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:08.970789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:08.970789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:13.665015 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:13.665933 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:13.665933 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:13.667095 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:13.698628 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:13.698971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.699734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:13.699734 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:13.699734 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:13.699734 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:13.699734 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:13.699734 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:13.699734 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.699734 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:13.700635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.700635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:13.700635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:13.700635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:13.700635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.700635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:32:13.700635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:13.700635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:13.700635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:13.700635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:13.700635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:13.700635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:13.701716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:13.701716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:13.701716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:13.701716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:13.701716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:13.701716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:13.701716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:13.701716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:13.701716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:13.701716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:13.701716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:13.701716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:13.701716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:13.702574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.702574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:13.702574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.702574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:13.702574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:13.702574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:13.702574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:13.702574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:13.702574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.702574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:13.702574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:13.702574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.702574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:13.702574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:13.702574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.703456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:13.703456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:13.703456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.703456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:32:13.703456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:13.703456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:13.703456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:13.703456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:13.703456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:13.703456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:13.703456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:13.703456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:13.703456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:13.703456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:13.703456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:13.703456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:13.703456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:13.703456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:13.703456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:13.704437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:13.704437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:13.704437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:13.704437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:13.704437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.704437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:13.900515 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:13.901110 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:13.901377 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:13.902491 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:13.902808 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:13.903099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.904002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:13.904002 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:13.904002 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:13.904002 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:13.904002 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:13.904002 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:13.904002 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.904002 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:13.905406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.905406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:13.905406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:13.905406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:13.905406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.905406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:32:13.905406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:13.905406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:13.905406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:13.905406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:13.905406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:13.905406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:13.905775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:13.905775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:13.905775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:13.905775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:13.905775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:13.905775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:13.905775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:13.905775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:13.905775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:13.905775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:13.905775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:13.905775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:13.905775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:13.906095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.906095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:13.907194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.907194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:13.907194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:13.907194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:13.907194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:13.907194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:13.907194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.907194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:13.907194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:13.907194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.907194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:13.907194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:13.907194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.907194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:13.907194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:13.907194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.907644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:32:13.907644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:13.907644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:13.907644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:13.907644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:13.907644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:13.907644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:13.907644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:13.907644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:13.907644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:13.907644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:13.907644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:13.907644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:13.907644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:13.907644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:13.907644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:13.907644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:13.907644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:13.907644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:13.908203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:13.908203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.908203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:13.951658 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:13.952176 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:13.952374 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:13.952561 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:32:13.953187 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:13.953515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.954446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:13.954446 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:13.954446 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:13.954446 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:13.954446 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:13.954446 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:13.954446 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.954446 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:13.955665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.955665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:32:13.955665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:13.955665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:13.955665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:13.955665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:13.955665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:13.955665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:13.955665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:13.955665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:13.955665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:13.955665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:32:13.955665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:13.955665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:32:13.955665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:32:13.956427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:13.956427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:32:13.956427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:13.956427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:32:13.956427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:32:13.956427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:13.956427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:32:13.956427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.956427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:32:13.957161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.957161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:13.957161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:13.957161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:13.957161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.957161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:32:13.957161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:32:13.957161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:32:13.957161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:13.957161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:13.957161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:13.957161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:13.957956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:13.957956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:13.957956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:13.957956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:13.957956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:13.957956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:13.957956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:13.957956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:13.957956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:13.957956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:13.957956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:13.957956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:13.957956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:13.957956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:13.958712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:13.958712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:13.958712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.958712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:13.959107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.959107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:13.959107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:13.959107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:13.959107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:13.959107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:13.959107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.959107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:13.959107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:13.959107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.959107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:13.959107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:13.959107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.959107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:13.959107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:13.959107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.959813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:32:13.959813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:32:13.959813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:32:13.959813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:13.959813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:13.959813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:13.959813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:13.959813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:13.959813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:13.959813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:13.959813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:13.959813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:13.959813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:13.959813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:13.959813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:13.959813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:13.959813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:13.959813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:13.960633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:13.960633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:13.960633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:13.960633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:13.960633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:13.960633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.960633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:13.960633 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:13.960633 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:13.960633 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:13.961188 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:13.961188 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:13.961188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.962175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:13.962175 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:13.962175 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:13.962175 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:13.962175 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:13.962175 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:13.962175 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.962175 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:13.963400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.963400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:32:13.963400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:13.963400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:13.963400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:13.963400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:13.963400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:13.963400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:13.963400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:13.963400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:13.963400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:13.963400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:32:13.963400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:13.963400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:32:13.963400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:32:13.964294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:13.964294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:32:13.964294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:13.964294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:32:13.964294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:32:13.964294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:13.964294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:32:13.964294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.964294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:32:13.964294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.964294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:13.964294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:13.964294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:13.964294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.965205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:32:13.965205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:32:13.965205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:13.965205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:13.965205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:13.965205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:13.965205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:13.965205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:13.965205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:13.965205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:13.965205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:13.965205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:13.965205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:13.965843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:13.965843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:13.965843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:13.965843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:13.965843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:13.965843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:13.965843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:13.965843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:13.965843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:13.965843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.965843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:13.965843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.966764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:13.966764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:13.966764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:13.966764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:13.966764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:13.966764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.966764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:13.966764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:13.966764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.966764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:13.966764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:13.966764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.966764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:13.966764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:13.966764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.966764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:32:13.966764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:32:13.966764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:13.966764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:13.967385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:13.967385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:13.967385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:13.967385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:13.967385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:13.967385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:13.967385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:13.967385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:13.967385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:13.967385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:13.967385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:13.967385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:13.967385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:13.967385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:13.967385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:13.967385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:13.967385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:13.967385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:13.967385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.967385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:13.967831 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:13.967831 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:13.967831 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:13.967831 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:13.967831 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:13.967831 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:13.967995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.967995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:13.967995 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:13.967995 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:13.967995 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:13.967995 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:13.967995 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:13.967995 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.967995 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:13.968479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.968479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:13.968479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:13.968479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:13.968479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.968479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:32:13.968479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:13.968479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:13.968479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:13.968479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:13.968479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:13.968479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:13.968746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:13.968746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:13.968746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:13.968746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:13.968746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:13.968746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:13.968746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:13.968746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:13.968746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:13.968746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:13.968746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:13.968746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:13.968746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:13.969001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.969001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:13.969435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.969435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:13.969435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:13.969435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:13.969435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:13.969435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:13.969435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.969435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:13.969435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:13.969435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.969435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:13.969435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:13.969435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.969435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:13.969435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:13.969435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:13.969759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:32:13.969759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:13.969759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:13.969759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:13.969759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:13.969759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:13.969759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:13.969759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:13.969759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:13.969759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:13.969759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:13.969759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:13.969759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:13.969759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:13.969759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:13.969759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:13.969759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:13.969759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:13.969759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:13.970145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:13.970145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:13.970145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:18.665604 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:18.666362 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:18.666530 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:18.667587 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:18.700110 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:18.700457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.701312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:18.701312 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:18.701312 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:18.701312 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:18.701312 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:18.701312 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:18.701312 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.701312 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:18.702732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.702732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:18.702732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:18.702732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:18.702732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.702732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:32:18.702732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:18.702732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:18.702732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:18.702732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:18.702732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:18.702732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:18.703068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:18.703068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:18.703068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:18.703068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:18.703068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:18.703068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:18.703068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:18.703068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:18.703068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:18.703068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:18.703068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:18.703068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:18.703068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:18.703393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.703393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:18.703393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.703393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:18.703393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:18.703393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:18.703393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:18.703393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:18.703393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.703393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:18.703393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:18.703393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.703393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:18.703393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:18.703393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.703821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:18.703821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:18.703821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.703821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:32:18.703821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:18.703821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:18.703821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:18.703821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:18.703821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:18.703821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:18.703821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:18.703821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:18.703821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:18.703821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:18.703821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:18.703821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:18.703821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:18.703821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:18.703821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:18.704278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:18.704278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:18.704278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:18.704278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:18.704278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.704278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:18.901225 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:18.901949 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:18.902252 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:18.903261 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:18.903490 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:18.903629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.904337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:18.904337 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:18.904337 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:18.904337 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:18.904337 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:18.904337 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:18.904337 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.904337 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:18.905689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.905689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:18.905689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:18.905689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:18.905689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.905689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:32:18.905689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:18.905689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:18.905689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:18.905689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:18.905689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:18.905689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:18.906287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:18.906287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:18.906287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:18.906287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:18.906287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:18.906287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:18.906287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:18.906287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:18.906287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:18.906287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:18.906287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:18.906287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:18.906287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:18.906976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.906976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:18.907229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.907229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:18.907229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:18.907229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:18.907229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:18.907229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:18.907229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.907229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:18.907229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:18.907229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.907229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:18.907229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:18.907229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.907229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:18.907229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:18.907229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.908081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:32:18.908081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:18.908081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:18.908081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:18.908081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:18.908081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:18.908081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:18.908081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:18.908081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:18.908081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:18.908081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:18.908081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:18.908081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:18.908081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:18.908081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:18.908081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:18.908081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:18.908081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:18.908081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:18.908957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:18.908957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.908957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:18.952401 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:18.952807 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:18.952930 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:18.953096 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:32:18.953556 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:18.953782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.954334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:18.954334 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:18.954334 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:18.954334 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:18.954334 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:18.954334 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:18.954334 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.954334 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:18.955195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.955195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:32:18.955195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:18.955195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:18.955195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:18.955195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:18.955195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:18.955195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:18.955195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:18.955195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:18.955195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:18.955195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:32:18.955195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:18.955195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:32:18.955195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:32:18.956015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:18.956015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:32:18.956015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:18.956015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:32:18.956015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:32:18.956015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:18.956015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:32:18.956015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.956015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:32:18.956455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.956455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:18.956455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:18.956455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:18.956455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.956455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:32:18.956455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:32:18.956455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:32:18.956455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:18.956455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:18.956455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:18.956455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:18.957103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:18.957103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:18.957103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:18.957103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:18.957103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:18.957103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:18.957103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:18.957103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:18.957103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:18.957103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:18.957103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:18.957103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:18.957103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:18.957103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:18.957883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:18.957883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:18.957883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.957883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:18.957883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.957883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:18.957883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:18.957883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:18.957883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:18.957883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:18.957883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.957883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:18.957883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:18.957883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.958666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:18.958666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:18.958666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.958666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:18.958666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:18.958666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.958666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:32:18.958666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:32:18.958666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:32:18.958666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:18.958666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:18.958666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:18.958666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:18.958666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:18.958666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:18.958666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:18.958666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:18.958666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:18.958666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:18.959639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:18.959639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:18.959639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:18.959639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:18.959639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:18.959639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:18.959639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:18.959639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:18.959639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:18.959639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:18.959639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.959639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:18.960499 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:18.960499 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:18.960499 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:18.960499 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:18.960897 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:18.960897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.961341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:18.961341 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:18.961341 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:18.961341 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:18.961341 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:18.961341 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:18.961341 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.961341 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:18.962083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.962083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:32:18.962083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:18.962083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:18.962083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:18.962083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:18.962083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:18.962083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:18.962083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:18.962083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:18.962083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:18.962083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:32:18.962083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:18.962083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:32:18.962083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:32:18.962916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:18.962916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:32:18.962916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:18.962916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:32:18.962916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:32:18.962916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:18.962916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:32:18.962916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.962916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:32:18.963463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.963463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:18.963463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:18.963463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:18.963463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.963463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:32:18.963463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:32:18.963463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:18.963463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:18.963463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:18.963463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:18.963982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:18.963982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:18.963982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:18.963982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:18.963982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:18.963982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:18.963982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:18.963982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:18.963982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:18.963982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:18.963982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:18.963982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:18.963982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:18.963982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:18.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:18.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:18.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:18.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:18.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:18.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:18.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:18.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:18.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:18.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:18.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.964865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:18.964865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:18.964865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.964865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:18.964865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:18.964865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.964865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:32:18.964865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:32:18.964865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:18.964865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:18.964865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:18.964865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:18.964865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:18.964865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:18.964865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:18.964865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:18.964865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:18.964865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:18.964865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:18.965227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:18.965227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:18.965227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:18.965227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:18.965227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:18.965227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:18.965227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:18.965227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:18.965227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:18.965227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.965227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:18.966657 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:18.967018 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:18.967177 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:18.967337 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:18.967592 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:18.967832 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:18.968163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.968773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:18.968773 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:18.968773 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:18.968773 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:18.968773 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:18.968773 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:18.968773 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.968773 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:18.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:18.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:18.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:18.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:32:18.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:18.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:18.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:18.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:18.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:18.969629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:18.970209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:18.970209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:18.970209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:18.970209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:18.970209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:18.970209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:18.970209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:18.970209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:18.970209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:18.970209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:18.970209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:18.970209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:18.970209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:18.970812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.970812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:18.970812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.970812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:18.970812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:18.970812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:18.970812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:18.970812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:18.970812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.970812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:18.970812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:18.970812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.970812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:18.970812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:18.970812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:18.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:18.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:18.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:32:18.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:18.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:18.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:18.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:18.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:18.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:18.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:18.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:18.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:18.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:18.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:18.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:18.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:18.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:18.971519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:18.972390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:18.972390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:18.972390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:18.972390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:18.972390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:18.972390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:23.666591 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:23.667344 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:23.667513 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:23.669144 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:23.715729 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:23.715729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.715729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:23.715729 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:23.715729 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:23.715729 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:23.715729 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:23.715729 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:23.715729 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.715729 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:23.716379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.047 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.716379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:23.716379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:23.716379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:23.716379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.716379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:32:23.716379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:23.716379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:23.716379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:23.716379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:23.716379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:23.716379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:23.716845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:23.716845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:23.716845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:23.716845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:23.716845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:23.716845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:23.716845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:23.716845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:23.716845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:23.716845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:23.716845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:23.716845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:23.716845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:23.717310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.717310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:23.717560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.717560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:23.717560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:23.717560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:23.717560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:23.717560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:23.717560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.717560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:23.717560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:23.717560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.717560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:23.717560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:23.717560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.717560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:23.717560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:23.717560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.718006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:32:23.718006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:23.718006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:23.718006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:23.718006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:23.718006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:23.718006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:23.718006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:23.718006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:23.718006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:23.718006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:23.718006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:23.718006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:23.718006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:23.718006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:23.718006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:23.718006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:23.718006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:23.718006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:23.718435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:23.718435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.718435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:23.901827 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:23.902438 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:23.902616 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:23.903804 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:23.904121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:23.904362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.905531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:23.905531 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:23.905531 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:23.905531 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:23.905531 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:23.905531 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:23.905531 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.905531 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:23.907107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.907107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:23.907107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:23.907107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:23.907107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.907107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:32:23.907107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:23.907107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:23.907107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:23.907107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:23.907107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:23.907107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:23.907665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:23.907665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:23.907665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:23.907665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:23.907665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:23.907665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:23.907665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:23.907665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:23.907665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:23.907665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:23.907665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:23.907665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:23.907665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:23.908097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.908097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:23.909457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.909457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:23.909457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:23.909457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:23.909457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:23.909457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:23.909457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.909457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:23.909457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:23.909457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.909457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:23.909457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:23.909457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.909457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:23.909457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:23.909457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.910014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:32:23.910014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:23.910014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:23.910014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:23.910014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:23.910014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:23.910014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:23.910014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:23.910014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:23.910014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:23.910014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:23.910014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:23.910014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:23.910014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:23.910014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:23.910014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:23.910014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:23.910014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:23.910014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:23.910715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:23.910715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.910715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:23.952771 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:23.953112 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:23.953245 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:23.953419 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:32:23.953933 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:23.954244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.955050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:23.955050 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:23.955050 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:23.955050 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:23.955050 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:23.955050 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:23.955050 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.955050 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:23.956108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.956108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:32:23.956108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:23.956108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:23.956108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:23.956108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:23.956108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:23.956108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:23.956108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:23.956108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:23.956108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:23.956108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:32:23.956108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:23.956108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:32:23.956108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:32:23.956918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:23.956918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:32:23.956918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:23.956918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:32:23.956918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:32:23.956918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:23.956918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:32:23.956918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.956918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:32:23.957358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.957358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:23.957358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:23.957358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:23.957358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.957358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:32:23.957358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:32:23.957358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:32:23.957358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:23.957358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:23.957358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:23.957358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:23.957947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:23.957947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:23.957947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:23.957947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:23.957947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:23.957947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:23.957947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:23.957947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:23.957947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:23.957947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:23.957947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:23.957947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:23.957947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:23.957947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:23.958648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:23.958648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:23.958648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.958648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:23.958648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.958648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:23.958648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:23.958648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:23.958648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:23.958648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:23.958648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.958648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:23.958648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:23.958648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.959317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:23.959317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:23.959317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.959317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:23.959317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:23.959317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.959317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:32:23.959317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:32:23.959317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:32:23.959317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:23.959317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:23.959317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:23.959317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:23.959317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:23.959317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:23.959317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:23.959317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:23.959317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:23.959317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:23.960197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:23.960197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:23.960197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:23.960197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:23.960197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:23.960197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:23.960197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:23.960197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:23.960197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:23.960197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:23.960197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.960197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:23.960711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:23.960711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:23.960711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:23.960893 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:23.961314 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:23.961702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.962363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:23.962363 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:23.962363 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:23.962363 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:23.962363 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:23.962363 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:23.962363 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.962363 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:23.963362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.963362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:32:23.963362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:23.963362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:23.963362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:23.963362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:23.963362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:23.963362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:23.963362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:23.963362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:23.963362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:23.963362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:32:23.963362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:23.963362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:32:23.963362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:32:23.963806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:23.963806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:32:23.963806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:23.963806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:32:23.963806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:32:23.963806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:23.963806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:32:23.963806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.963806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:32:23.964550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.964550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:23.964550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:23.964550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:23.964550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.964550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:32:23.964550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:32:23.964550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:23.964550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:23.964550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:23.964550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:23.964897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:23.964897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:23.964897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:23.964897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:23.964897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:23.964897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:23.964897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:23.964897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:23.964897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:23.964897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:23.964897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:23.964897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:23.964897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:23.964897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:23.965324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:23.965324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:23.965324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.965324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:23.965961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.965961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:23.965961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:23.965961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:23.965961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:23.965961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:23.965961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.965961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:23.965961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:23.965961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.965961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:23.965961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:23.965961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.965961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:23.965961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:23.965961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.965961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:32:23.966525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:32:23.966525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:23.966525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:23.966525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:23.966525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:23.966525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:23.966525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:23.966525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:23.966525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:23.966525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:23.966525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:23.966525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:23.966525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:23.966525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:23.966525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:23.966525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:23.966525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:23.966525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:23.966525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:23.967177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:23.967177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:23.967177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.967177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:23.967177 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:23.967379 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:23.967430 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:23.967619 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:23.968029 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:23.968423 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:23.968645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.969288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:23.969288 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:23.969288 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:23.969288 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:23.969288 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:23.969288 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:23.969288 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.969288 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:23.970167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.970167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:23.970167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:23.970167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:23.970167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.970167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:32:23.970167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:23.970167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:23.970167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:23.970167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:23.970167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:23.970167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:23.970619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:23.970619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:23.970619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:23.970619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:23.970619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:23.970619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:23.970619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:23.970619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:23.970619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:23.970619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:23.970619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:23.970619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:23.970619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:23.971008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.971008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:23.971155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.971155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:23.971155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:23.971155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:23.971155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:23.971155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:23.971155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.971155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:23.971155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:23.971155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.971155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:23.971155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:23.971155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.971155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:23.971155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:23.971155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:23.971625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:32:23.971625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:23.971625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:23.971625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:23.971625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:23.971625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:23.971625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:23.971625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:23.971625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:23.971625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:23.971625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:23.971625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:23.971625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:23.971625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:23.971625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:23.971625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:23.971625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:23.971625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:23.971625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:23.972203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:23.972203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:23.972203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:28.667666 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:28.668546 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:28.668546 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:28.669584 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:28.708072 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:28.708446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.709189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:28.709189 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:28.709189 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:28.709189 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:28.709189 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:28.709189 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:28.709189 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.709189 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:28.709839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.709839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:28.709839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:28.709839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:28.709839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.709839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:32:28.709839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:28.709839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:28.709839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:28.709839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:28.709839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:28.709839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:28.710422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:28.710422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:28.710422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:28.710422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:28.710422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:28.710422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:28.710422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:28.710422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:28.710422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:28.710422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:28.710422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:28.710422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:28.710422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:28.711008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.711008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:28.711008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.711008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:28.711008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:28.711008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:28.711008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:28.711008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:28.711008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.711008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:28.711008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:28.711008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.711008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:28.711008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:28.711008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.711664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:28.711664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:28.711664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.711664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:32:28.711664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:28.711664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:28.711664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:28.711664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:28.711664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:28.711664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:28.711664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:28.711664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:28.711664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:28.711664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:28.711664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:28.711664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:28.711664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:28.711664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:28.711664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:28.712471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:28.712471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:28.712471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:28.712471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:28.712471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.712471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:28.902380 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:28.902754 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:28.902958 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:28.903845 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:28.904121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:28.904342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.905100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:28.905100 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:28.905100 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:28.905100 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:28.905100 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:28.905100 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:28.905100 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.905100 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:28.906457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.906457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:28.906457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:28.906457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:28.906457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.906457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:32:28.906457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:28.906457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:28.906457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:28.906457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:28.906457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:28.906457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:28.906772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:28.906772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:28.906772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:28.906772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:28.906772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:28.906772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:28.906772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:28.906772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:28.906772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:28.906772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:28.906772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:28.906772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:28.906772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:28.907066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.907066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:28.908185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.908185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:28.908185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:28.908185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:28.908185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:28.908185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:28.908185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.908185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:28.908185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:28.908185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.908185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:28.908185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:28.908185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.908185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:28.908185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:28.908185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.908571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:32:28.908571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:28.908571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:28.908571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:28.908571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:28.908571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:28.908571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:28.908571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:28.908571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:28.908571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:28.908571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:28.908571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:28.908571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:28.908571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:28.908571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:28.908571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:28.908571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:28.908571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:28.908571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:28.908998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:28.908998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.908998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:28.953238 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:28.953539 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:28.953751 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:28.953933 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:32:28.954366 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:28.954643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.955391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:28.955391 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:28.955391 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:28.955391 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:28.955391 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:28.955391 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:28.955391 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.955391 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:28.956533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.956533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:32:28.956533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:28.956533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:28.956533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:28.956533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:28.956533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:28.956533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:28.956533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:28.956533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:28.956533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:28.956533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:32:28.956533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:28.956533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:32:28.956533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:32:28.956882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:28.956882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:32:28.956882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:28.956882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:32:28.956882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:32:28.956882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:28.956882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:32:28.956882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.956882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:32:28.957837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.957837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:28.957837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:28.957837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:28.957837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.957837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:32:28.957837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:32:28.957837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:32:28.957837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:28.957837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:28.957837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:28.957837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:28.958179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:28.958179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:28.958179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:28.958179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:28.958179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:28.958179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:28.958179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:28.958179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:28.958179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:28.958179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:28.958179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:28.958179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:28.958179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:28.958179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:28.958428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:28.958428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:28.958428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.958428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:28.959594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.959594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:28.959594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:28.959594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:28.959594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:28.959594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:28.959594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.959594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:28.959594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:28.959594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.959594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:28.959594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:28.959594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.959594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:28.959594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:28.959594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.959929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:32:28.959929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:32:28.959929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:32:28.959929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:28.959929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:28.959929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:28.959929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:28.959929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:28.959929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:28.959929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:28.959929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:28.959929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:28.959929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:28.959929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:28.959929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:28.959929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:28.959929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:28.959929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:28.960275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:28.960275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:28.960275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:28.960275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:28.960275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:28.960275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.960275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:28.961161 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:28.961527 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:28.961633 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:28.961777 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:28.962199 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:28.962563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.963260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:28.963260 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:28.963260 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:28.963260 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:28.963260 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:28.963260 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:28.963260 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.963260 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:28.963973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.963973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:32:28.963973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:28.963973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:28.963973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:28.963973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:28.963973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:28.963973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:28.963973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:28.963973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:28.963973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:28.963973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:32:28.963973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:28.963973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:32:28.963973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:32:28.964273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:28.964273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:32:28.964273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:28.964273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:32:28.964273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:32:28.964273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:28.964273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:32:28.964273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.964273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:32:28.964849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.964849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:28.964849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:28.964849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:28.964849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.964849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:32:28.964849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:32:28.964849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:28.964849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:28.964849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:28.964849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:28.965122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:28.965122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:28.965122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:28.965122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:28.965122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:28.965122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:28.965122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:28.965122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:28.965122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:28.965122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:28.965122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:28.965122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:28.965122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:28.965122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:28.965388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:28.965388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:28.965388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.965388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:28.965879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.965879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:28.965879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:28.965879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:28.965879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:28.965879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:28.965879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.965879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:28.965879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:28.965879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.965879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:28.965879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:28.965879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.965879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:28.965879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:28.965879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.965879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:32:28.966278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:32:28.966278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:28.966278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:28.966278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:28.966278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:28.966278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:28.966278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:28.966278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:28.966278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:28.966278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:28.966278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:28.966278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:28.966278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:28.966278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:28.966278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:28.966278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:28.966278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:28.966278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:28.966278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:28.966626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:28.966626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:28.966626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.966626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:28.967356 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:28.967643 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:28.967771 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:28.967920 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:28.968399 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:28.968806 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:28.969035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.969657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:28.969657 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:28.969657 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:28.969657 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:28.969657 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:28.969657 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:28.969657 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.969657 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:28.970426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.970426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:28.970426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:28.970426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:28.970426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.970426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:32:28.970426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:28.970426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:28.970426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:28.970426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:28.970426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:28.970426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:28.970686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:28.970686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:28.970686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:28.970686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:28.970686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:28.970686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:28.970686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:28.970686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:28.970686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:28.970686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:28.970686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:28.970686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:28.970686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:28.970946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.970946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:28.971391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.971391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:28.971391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:28.971391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:28.971391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:28.971391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:28.971391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.971391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:28.971391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:28.971391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.971391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:28.971391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:28.971391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.971391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:28.971391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:28.971391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:28.971727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:32:28.971727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:28.971727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:28.971727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:28.971727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:28.971727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:28.971727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:28.971727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:28.971727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:28.971727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:28.971727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:28.971727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:28.971727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:28.971727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:28.971727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:28.971727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:28.971727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:28.971727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:28.971727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:28.972094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:28.972094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:28.972094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:33.668560 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:33.669323 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:33.669503 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:33.670826 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:33.705895 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:33.705895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.706371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:33.706371 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:33.706371 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:33.706371 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:33.706371 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:33.706371 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:33.706371 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.706371 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:33.707148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.707148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:33.707148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:33.707148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:33.707148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.707148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:32:33.707148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:33.707148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:33.707148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:33.707148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:33.707148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:33.707148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:33.707620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:33.707620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:33.707620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:33.707620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:33.707620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:33.707620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:33.707620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:33.707620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:33.707620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:33.707620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:33.707620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:33.707620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:33.707620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:33.708039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.708039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:33.708039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.708039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:33.708039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:33.708039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:33.708039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:33.708039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:33.708039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.708039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:33.708039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:33.708039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.708039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:33.708039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:33.708039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.708504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:33.708504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:33.708504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.708504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:32:33.708504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:33.708504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:33.708504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:33.708504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:33.708504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:33.708504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:33.708504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:33.708504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:33.708504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:33.708504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:33.708504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:33.708504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:33.708504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:33.708504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:33.708504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:33.709018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:33.709018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:33.709018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:33.709018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:33.709018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.709018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:33.903139 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:33.903897 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:33.904089 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:33.905249 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:33.905476 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:33.905636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.906425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:33.906425 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:33.906425 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:33.906425 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:33.906425 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:33.906425 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:33.906425 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.906425 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:33.907940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.907940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:33.907940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:33.907940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:33.907940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.907940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:32:33.907940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:33.907940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:33.907940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:33.907940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:33.907940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:33.907940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:33.908631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:33.908631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:33.908631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:33.908631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:33.908631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:33.908631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:33.908631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:33.908631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:33.908631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:33.908631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:33.908631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:33.908631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:33.908631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:33.909274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.909274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:33.909601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.909601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:33.909601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:33.909601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:33.909601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:33.909601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:33.909601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.909601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:33.909601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:33.909601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.909601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:33.909601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:33.909601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.909601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:33.909601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:33.909601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.910427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:32:33.910427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:33.910427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:33.910427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:33.910427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:33.910427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:33.910427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:33.910427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:33.910427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:33.910427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:33.910427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:33.910427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:33.910427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:33.910427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:33.910427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:33.910427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:33.910427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:33.910427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:33.910427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:33.911345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:33.911345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.911345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:33.953979 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:33.954392 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:33.954574 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:33.954811 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:32:33.955256 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:33.955468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.956106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:33.956106 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:33.956106 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:33.956106 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:33.956106 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:33.956106 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:33.956106 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.956106 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:33.957394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.957394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:32:33.957394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:33.957394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:33.957394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:33.957394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:33.957394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:33.957394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:33.957394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:33.957394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:33.957394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:33.957394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:32:33.957394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:33.957394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:32:33.957394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:32:33.958123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:33.958123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:32:33.958123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:33.958123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:32:33.958123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:32:33.958123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:33.958123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:32:33.958123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.958123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:32:33.958780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.958780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:33.958780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:33.958780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:33.958780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.958780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:32:33.958780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:32:33.958780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:32:33.958780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:33.958780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:33.958780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:33.958780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:33.959357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:33.959357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:33.959357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:33.959357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:33.959357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:33.959357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:33.959357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:33.959357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:33.959357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:33.959357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:33.959357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:33.959357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:33.959357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:33.959357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:33.960015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:33.960015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:33.960015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.960015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:33.960562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.960562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:33.960562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:33.960562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:33.960562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:33.960562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:33.960562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.960562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:33.960562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:33.960562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.960562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:33.960562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:33.960562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.960562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:33.960562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:33.960562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.961300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:32:33.961300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:32:33.961300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:32:33.961300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:33.961300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:33.961300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:33.961300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:33.961300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:33.961300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:33.961300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:33.961300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:33.961300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:33.961300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:33.961300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:33.961300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:33.961300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:33.961300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:33.961300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:33.962053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:33.962053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:33.962053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:33.962053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:33.962053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:33.962053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.962053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:33.962053 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:33.962423 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:33.962423 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:33.962539 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:33.962887 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:33.963078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.963603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:33.963603 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:33.963603 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:33.963603 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:33.963603 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:33.963603 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:33.963603 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.963603 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:33.964440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.964440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:32:33.964440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:33.964440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:33.964440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:33.964440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:33.964440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:33.964440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:33.964440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:33.964440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:33.964440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:33.964440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:32:33.964440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:33.964440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:32:33.964440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:32:33.965109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:33.965109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:32:33.965109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:33.965109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:32:33.965109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:32:33.965109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:33.965109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:32:33.965109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.965109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:32:33.965483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.965483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:33.965483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:33.965483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:33.965483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.965483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:32:33.965483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:32:33.965483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:33.965483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:33.965483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:33.965483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:33.965980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:33.965980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:33.965980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:33.965980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:33.965980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:33.965980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:33.965980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:33.965980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:33.965980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:33.965980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:33.965980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:33.965980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:33.965980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:33.965980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:33.966537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:33.966537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:33.966537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.966537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:33.966735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.966735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:33.966735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:33.966735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:33.966735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:33.966735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:33.966735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.966735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:33.966735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:33.966735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.966735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:33.966735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:33.966735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.966735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:33.966735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:33.966735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.966735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:32:33.967497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:32:33.967497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:33.967497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:33.967497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:33.967497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:33.967497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:33.967497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:33.967497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:33.967497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:33.967497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:33.967497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:33.967497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:33.967497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:33.967497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:33.967497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:33.967497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:33.967497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:33.967497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:33.967497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:33.968447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:33.968447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:33.968447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.968447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:33.968447 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:33.968447 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:33.968447 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:33.968447 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:33.968818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:33.968912 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:33.969047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.969509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:33.969509 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:33.969509 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:33.969509 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:33.969509 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:33.969509 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:33.969509 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.969509 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:33.970393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.970393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:33.970393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:33.970393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:33.970393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.970393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:32:33.970393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:33.970393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:33.970393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:33.970393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:33.970393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:33.970393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:33.970702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:33.970702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:33.970702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:33.970702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:33.970702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:33.970702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:33.970702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:33.970702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:33.970702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:33.970702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:33.970702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:33.970702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:33.970702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:33.970964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.970964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:33.971408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.971408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:33.971408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:33.971408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:33.971408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:33.971408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:33.971408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.971408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:33.971408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:33.971408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.971408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:33.971408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:33.971408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.971408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:33.971408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:33.971408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:33.971775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:32:33.971775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:33.971775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:33.971775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:33.971775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:33.971775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:33.971775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:33.971775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:33.971775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:33.971775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:33.971775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:33.971775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:33.971775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:33.971775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:33.971775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:33.971775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:33.971775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:33.971775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:33.971775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:33.972185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:33.972185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:33.972185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:38.669612 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:38.670627 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:38.670627 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:38.671942 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:38.708520 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:38.708919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.709828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:38.709828 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:38.709828 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:38.709828 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:38.709828 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:38.709828 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:38.709828 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.709828 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:38.710714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.710714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:38.710714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:38.710714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:38.710714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.710714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:32:38.710714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:38.710714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:38.710714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:38.710714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:38.710714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:38.710714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:38.711245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:38.711245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:38.711245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:38.711245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:38.711245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:38.711245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:38.711245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:38.711245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:38.711245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:38.711245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:38.711245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:38.711245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:38.711245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:38.711701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.711701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:38.711701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.711701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:38.711701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:38.711701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:38.711701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:38.711701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:38.711701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.711701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:38.711701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:38.711701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.711701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:38.711701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:38.711701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.712082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:38.712082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:38.712082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.712082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:32:38.712082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:38.712082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:38.712082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:38.712082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:38.712082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:38.712082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:38.712082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:38.712082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:38.712082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:38.712082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:38.712082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:38.712082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:38.712082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:38.712082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:38.712082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:38.712625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:38.712625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:38.712625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:38.712625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:38.712625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.712625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:38.904267 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:38.904862 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:38.905052 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:38.906768 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:38.908340 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:38.908340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.908649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:38.908649 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:38.908649 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:38.908649 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:38.908649 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:38.908649 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:38.908649 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.908649 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:38.910182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.910182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:38.910182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:38.910182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:38.910182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.910182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:32:38.910182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:38.910182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:38.910182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:38.910182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:38.910182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:38.910182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:38.910565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:38.910565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:38.910565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:38.910565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:38.910565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:38.910565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:38.910565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:38.910565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:38.910565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:38.910565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:38.910565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:38.910565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:38.910565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:38.910902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.910902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:38.911829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.911829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:38.911829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:38.911829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:38.911829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:38.911829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:38.911829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.911829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:38.911829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:38.911829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.911829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:38.911829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:38.911829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.911829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:38.911829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:38.911829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.912255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:32:38.912255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:38.912255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:38.912255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:38.912255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:38.912255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:38.912255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:38.912255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:38.912255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:38.912255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:38.912255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:38.912255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:38.912255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:38.912255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:38.912255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:38.912255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:38.912255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:38.912255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:38.912255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:38.912732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:38.912732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.912732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:38.954702 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:38.955130 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:38.955227 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:38.955359 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:32:38.956149 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:38.956515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.957394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:38.957394 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:38.957394 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:38.957394 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:38.957394 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:38.957394 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:38.957394 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.957394 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:38.958393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.958393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:32:38.958393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:38.958393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:38.958393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:38.958393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:38.958393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:38.958393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:38.958393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:38.958393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:38.958393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:38.958393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:32:38.958393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:38.958393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:32:38.958393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:32:38.959285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:38.959285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:32:38.959285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:38.959285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:32:38.959285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:32:38.959285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:38.959285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:32:38.959285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.959285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:32:38.959776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.959776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:38.959776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:38.959776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:38.959776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.959776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:32:38.959776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:32:38.959776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:32:38.959776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:38.959776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:38.959776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:38.959776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:38.960379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:38.960379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:38.960379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:38.960379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:38.960379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:38.960379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:38.960379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:38.960379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:38.960379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:38.960379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:38.960379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:38.960379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:38.960379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:38.960379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:38.961042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:38.961042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:38.961042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.961042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:38.961042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.961042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:38.961042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:38.961042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:38.961042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:38.961042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:38.961042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.961042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:38.961042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:38.961042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.961850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:38.961850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:38.961850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.961850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:38.961850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:38.961850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.961850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:32:38.961850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:32:38.961850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:32:38.961850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:38.961850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:38.961850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:38.961850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:38.961850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:38.961850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:38.961850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:38.961850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:38.961850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:38.961850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:38.962769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:38.962769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:38.962769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:38.962769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:38.962769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:38.962769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:38.962769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:38.962769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:38.962769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:38.962769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:38.962769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.962769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:38.962769 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:38.962769 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:38.963538 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:38.963538 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:38.963538 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:38.964133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.964926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:38.964926 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:38.964926 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:38.964926 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:38.964926 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:38.964926 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:38.964926 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.964926 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:38.965764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.965764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:32:38.965764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:38.965764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:38.965764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:38.965764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:38.965764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:38.965764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:38.965764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:38.965764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:38.965764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:38.965764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:32:38.965764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:38.965764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:32:38.965764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:32:38.966830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:38.966830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:32:38.966830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:38.966830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:32:38.966830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:32:38.966830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:38.966830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:32:38.966830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.966830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:32:38.966830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.966830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:38.966830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:38.966830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:38.966830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.967475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:32:38.967475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:32:38.967475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:38.967475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:38.967475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:38.967475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:38.967475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:38.967475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:38.967475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:38.967475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:38.967475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:38.967475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:38.967475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:38.968095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:38.968095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:38.968095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:38.968095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:38.968095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:38.968095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:38.968095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:38.968095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:38.968095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:38.968095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.968095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:38.968095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.968771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:38.968771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:38.968771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:38.968771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:38.968771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:38.968771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.968771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:38.968771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:38.968771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.968771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:38.968771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:38.968771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.968771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:38.968771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:38.968771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.968771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:32:38.968771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:32:38.968771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:38.968771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:38.970023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:38.970023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:38.970023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:38.970023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:38.970023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:38.970023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:38.970023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:38.970023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:38.970023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:38.970023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:38.970023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:38.970023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:38.970023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:38.970023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:38.970023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:38.970023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:38.970023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:38.970023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:38.970023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.970023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:38.971168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:38.971168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:38.971168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:38.971168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:38.971168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:38.971168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:38.971336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.971336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:38.971336 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:38.971336 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:38.971336 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:38.971336 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:38.971336 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:38.971336 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.971336 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:38.972213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.972213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:38.972213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:38.972213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:38.972213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.972213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:32:38.972213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:38.972213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:38.972213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:38.972213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:38.972213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:38.972213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:38.972511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:38.972511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:38.972511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:38.972511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:38.972511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:38.972511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:38.972511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:38.972511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:38.972511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:38.972511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:38.972511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:38.972511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:38.972511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:38.972802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.972802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:38.973337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.973337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:38.973337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:38.973337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:38.973337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:38.973337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:38.973337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.973337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:38.973337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:38.973337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.973337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:38.973337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:38.973337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.973337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:38.973337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:38.973337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:38.973723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:32:38.973723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:38.973723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:38.973723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:38.973723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:38.973723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:38.973723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:38.973723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:38.973723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:38.973723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:38.973723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:38.973723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:38.973723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:38.973723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:38.973723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:38.973723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:38.973723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:38.973723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:38.973723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:38.974162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:38.974162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:38.974162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:43.670438 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:43.671315 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:43.671389 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:43.672595 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:43.703441 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:43.703922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.704902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:43.704902 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:43.704902 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:43.704902 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:43.704902 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:43.704902 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:43.704902 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.704902 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:43.705798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.705798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:43.705798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:43.705798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:43.705798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.705798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:32:43.705798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:43.705798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:43.705798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:43.705798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:43.705798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:43.705798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:43.706473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:43.706473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:43.706473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:43.706473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:43.706473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:43.706473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:43.706473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:43.706473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:43.706473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:43.706473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:43.706473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:43.706473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:43.706473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:43.707176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.707176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:43.707176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.707176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:43.707176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:43.707176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:43.707176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:43.707176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:43.707176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.707176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:43.707176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:43.707176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.707176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:43.707176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:43.707176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.708057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:43.708057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:43.708057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.708057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:32:43.708057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:43.708057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:43.708057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:43.708057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:43.708057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:43.708057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:43.708057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:43.708057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:43.708057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:43.708057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:43.708057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:43.708057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:43.708057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:43.708057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:43.708057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:43.709165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:43.709165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:43.709165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:43.709165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:43.709165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.709165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:43.905364 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:43.906195 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:43.906412 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:43.907597 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:43.907922 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:43.908087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.908906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:43.908906 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:43.908906 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:43.908906 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:43.908906 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:43.908906 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:43.908906 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.908906 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:43.910634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.910634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:43.910634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:43.910634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:43.910634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.910634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:32:43.910634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:43.910634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:43.910634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:43.910634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:43.910634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:43.910634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:43.911031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:43.911031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:43.911031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:43.911031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:43.911031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:43.911031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:43.911031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:43.911031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:43.911031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:43.911031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:43.911031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:43.911031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:43.911031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:43.911404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.911404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:43.912570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.912570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:43.912570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:43.912570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:43.912570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:43.912570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:43.912570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.912570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:43.912570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:43.912570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.912570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:43.912570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:43.912570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.912570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:43.912570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:43.912570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.913092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:32:43.913092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:43.913092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:43.913092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:43.913092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:43.913092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:43.913092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:43.913092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:43.913092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:43.913092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:43.913092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:43.913092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:43.913092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:43.913092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:43.913092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:43.913092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:43.913092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:43.913092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:43.913092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:43.913646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:43.913646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.913646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:43.955442 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:43.956000 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:43.956109 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:43.956227 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:32:43.956944 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:43.957332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.958192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:43.958192 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:43.958192 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:43.958192 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:43.958192 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:43.958192 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:43.958192 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.958192 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:43.958964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.958964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:32:43.958964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:43.958964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:43.958964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:43.958964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:43.958964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:43.958964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:43.958964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:43.958964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:43.958964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:43.958964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:32:43.958964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:43.958964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:32:43.958964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:32:43.959434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:43.959434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:32:43.959434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:43.959434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:32:43.959434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:32:43.959434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:43.959434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:32:43.959434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.959434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:32:43.959783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.959783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:43.959783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:43.959783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:43.959783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.959783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:32:43.959783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:32:43.959783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:32:43.959783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:43.959783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:43.959783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:43.959783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:43.960137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:43.960137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:43.960137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:43.960137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:43.960137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:43.960137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:43.960137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:43.960137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:43.960137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:43.960137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:43.960137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:43.960137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:43.960137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:43.960137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:43.960511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:43.960511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:43.960511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.960511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:43.960843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.960843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:43.960843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:43.960843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:43.960843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:43.960843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:43.960843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.960843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:43.960843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:43.960843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.960843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:43.960843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:43.960843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.960843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:43.960843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:43.960843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.961299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:32:43.961299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:32:43.961299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:32:43.961299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:43.961299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:43.961299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:43.961299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:43.961299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:43.961299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:43.961299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:43.961299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:43.961299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:43.961299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:43.961299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:43.961299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:43.961299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:43.961299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:43.961299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:43.961819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:43.961819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:43.961819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:43.961819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:43.961819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:43.961819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.961819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:43.962800 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:43.963099 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:43.963196 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:43.963308 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:43.963630 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:43.963922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.964371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:43.964371 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:43.964371 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:43.964371 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:43.964371 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:43.964371 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:43.964371 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.964371 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:43.965072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.965072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:32:43.965072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:43.965072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:43.965072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:43.965072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:43.965072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:43.965072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:43.965072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:43.965072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:43.965072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:43.965072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:32:43.965072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:43.965072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:32:43.965072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:32:43.965472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:43.965472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:32:43.965472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:43.965472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:32:43.965472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:32:43.965472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:43.965472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:32:43.965472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.965472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:32:43.965947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.965947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:43.965947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:43.965947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:43.965947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.965947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:32:43.965947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:32:43.965947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:43.965947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:43.965947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:43.965947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:43.966268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:43.966268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:43.966268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:43.966268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:43.966268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:43.966268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:43.966268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:43.966268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:43.966268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:43.966268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:43.966268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:43.966268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:43.966268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:43.966268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:43.966641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:43.966641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:43.966641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.966641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:43.966961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.966961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:43.966961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:43.966961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:43.966961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:43.966961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:43.966961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.966961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:43.966961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:43.966961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.966961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:43.966961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:43.966961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.966961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:43.966961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:43.966961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.966961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:32:43.967438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:32:43.967438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:43.967438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:43.967438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:43.967438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:43.967438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:43.967438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:43.967438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:43.967438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:43.967438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:43.967438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:43.967438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:43.967438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:43.967438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:43.967438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:43.967438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:43.967438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:43.967438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:43.967438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:43.967953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:43.967953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:43.967953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.967953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:43.968838 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:43.969113 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:43.969208 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:43.969343 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:43.969634 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:43.969927 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:43.970111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.970576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:43.970576 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:43.970576 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:43.970576 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:43.970576 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:43.970576 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:43.970576 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.970576 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:43.971344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.971344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:43.971344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:43.971344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:43.971344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.971344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:32:43.971344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:43.971344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:43.971344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:43.971344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:43.971344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:43.971344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:43.971704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:43.971704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:43.971704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:43.971704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:43.971704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:43.971704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:43.971704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:43.971704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:43.971704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:43.971704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:43.971704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:43.971704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:43.971704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:43.972055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.972055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:43.972257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.972257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:43.972257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:43.972257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:43.972257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:43.972257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:43.972257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.972257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:43.972257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:43.972257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.972257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:43.972257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:43.972257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.972257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:43.972257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:43.972257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:43.972702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:32:43.972702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:43.972702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:43.972702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:43.972702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:43.972702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:43.972702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:43.972702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:43.972702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:43.972702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:43.972702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:43.972702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:43.972702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:43.972702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:43.972702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:43.972702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:43.972702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:43.972702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:43.972702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:43.973271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:43.973271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:43.973271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:48.671157 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:48.672167 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:48.672167 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:48.673442 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:48.705298 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:48.705709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.706538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:48.706538 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:48.706538 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:48.706538 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:48.706538 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:48.706538 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:48.706538 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.706538 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:48.707450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.707450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:48.707450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:48.707450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:48.707450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.707450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:32:48.707450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:48.707450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:48.707450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:48.707450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:48.707450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:48.707450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:48.708028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:48.708028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:48.708028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:48.708028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:48.708028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:48.708028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:48.708028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:48.708028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:48.708028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:48.708028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:48.708028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:48.708028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:48.708028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:48.708499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.708499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:48.708499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.708499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:48.708499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:48.708499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:48.708499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:48.708499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:48.708499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.708499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:48.708499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:48.708499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.708499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:48.708499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:48.708499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.708991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:48.708991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:48.708991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.708991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:32:48.708991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:48.708991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:48.708991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:48.708991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:48.708991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:48.708991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:48.708991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:48.708991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:48.708991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:48.708991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:48.708991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:48.708991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:48.708991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:48.708991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:48.708991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:48.709550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:48.709550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:48.709550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:48.709550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:48.709550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.709550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:48.906168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:48.906870 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:48.907081 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:48.909526 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:48.909711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:48.910220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.912589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:48.912589 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:48.912589 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:48.912589 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:48.912589 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:48.912589 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:48.912589 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.912589 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:48.914110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.914110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:48.914110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:48.914110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:48.914110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.914110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:32:48.914110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:48.914110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:48.914110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:48.914110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:48.914110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:48.914110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:48.914514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:48.914514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:48.914514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:48.914514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:48.914514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:48.914514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:48.914514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:48.914514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:48.914514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:48.914514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:48.914514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:48.914514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:48.914514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:48.914938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.914938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:48.915642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.915642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:48.915642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:48.915642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:48.915642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:48.915642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:48.915642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.915642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:48.915642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:48.915642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.915642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:48.915642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:48.915642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.915642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:48.915642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:48.915642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.916166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:32:48.916166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:48.916166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:48.916166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:48.916166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:48.916166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:48.916166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:48.916166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:48.916166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:48.916166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:48.916166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:48.916166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:48.916166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:48.916166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:48.916166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:48.916166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:48.916166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:48.916166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:48.916166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:48.916839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:48.916839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.916839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:48.956144 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:48.956412 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:48.956614 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:48.956939 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:32:48.957665 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:48.957989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.958791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:48.958791 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:48.958791 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:48.958791 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:48.958791 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:48.958791 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:48.958791 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.958791 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:48.960132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.960132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:32:48.960132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:48.960132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:48.960132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:48.960132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:48.960132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:48.960132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:48.960132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:48.960132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:48.960132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:48.960132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:32:48.960132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:48.960132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:32:48.960132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:32:48.960953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:48.960953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:32:48.960953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:48.960953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:32:48.960953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:32:48.960953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:48.960953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:32:48.960953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.960953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:32:48.961552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.961552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:48.961552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:48.961552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:48.961552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.961552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:32:48.961552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:32:48.961552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:32:48.961552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:48.961552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:48.961552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:48.961552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:48.962140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:48.962140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:48.962140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:48.962140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:48.962140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:48.962140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:48.962140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:48.962140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:48.962140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:48.962140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:48.962140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:48.962140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:48.962140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:48.962140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:48.962768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:48.962768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:48.962768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.962768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:48.963346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.963346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:48.963346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:48.963346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:48.963346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:48.963346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:48.963346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.963346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:48.963346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:48.963346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.963346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:48.963346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:48.963346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.963346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:48.963346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:48.963346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.964143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:32:48.964143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:32:48.964143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:32:48.964143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:48.964143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:48.964143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:48.964143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:48.964143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:48.964143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:48.964143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:48.964143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:48.964143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:48.964143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:48.964143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:48.964143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:48.964143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:48.964143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:48.964143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:48.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:48.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:48.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:48.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:48.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:48.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:48.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:48.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:48.964755 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:48.965079 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:48.965079 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:48.965283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.965971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:48.965971 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:48.965971 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:48.965971 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:48.965971 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:48.965971 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:48.965971 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.965971 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:48.966668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.966668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:32:48.966668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:48.966668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:48.966668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:48.966668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:48.966668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:48.966668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:48.966668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:48.966668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:48.966668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:48.966668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:32:48.966668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:48.966668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:32:48.966668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:32:48.967117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:48.967117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:32:48.967117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:48.967117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:32:48.967117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:32:48.967117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:48.967117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:32:48.967117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.967117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:32:48.967843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.967843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:48.967843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:48.967843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:48.967843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.967843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:32:48.967843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:32:48.967843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:48.967843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:48.967843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:48.967843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:48.968172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:48.968172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:48.968172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:48.968172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:48.968172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:48.968172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:48.968172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:48.968172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:48.968172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:48.968172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:48.968172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:48.968172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:48.968172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:48.968172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:48.970236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:48.970236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:48.970236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.970236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:48.970236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.970236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:48.970236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:48.970236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:48.970236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:48.970236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:48.970236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.970236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:48.970236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:48.970236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.970593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:48.970593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:48.970593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.970593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:48.970593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:48.970593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.970593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:32:48.970593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:32:48.970593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:48.970593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:48.970593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:48.970593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:48.970593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:48.970593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:48.970593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:48.970593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:48.970593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:48.970593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:48.970593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:48.971127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:48.971127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:48.971127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:48.971127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:48.971127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:48.971127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:48.971127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:48.971127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:48.971127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:48.971127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.971127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:48.971127 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:48.971127 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:48.971508 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:48.971508 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:48.971508 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:48.971508 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:48.971508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.971508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:48.971508 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:48.971508 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:48.971508 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:48.971796 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:48.971796 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:48.971796 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.971796 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:48.971796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.971796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:48.971796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:48.971796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:48.971796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.971796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:32:48.971796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:48.971796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:48.971796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:48.971796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:48.972195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:48.972195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:48.972195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:48.972195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:48.972195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:48.972195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:48.972195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:48.972195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:48.972195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:48.972195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:48.972195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:48.972195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:48.972195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:48.972195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:48.972568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:48.972568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.972568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:48.972656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.972656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:48.972656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:48.972656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:48.972656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:48.972656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:48.972656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.972656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:48.972656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:48.972656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.972656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:48.972656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:48.972656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.972656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:48.972656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:48.972656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:48.973128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:32:48.973128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:48.973128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:48.973128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:48.973128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:48.973128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:48.973128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:48.973128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:48.973128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:48.973128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:48.973128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:48.973128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:48.973128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:48.973128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:48.973128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:48.973128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:48.973128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:48.973128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:48.973128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:48.973631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:48.973631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:48.973631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:53.672007 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:53.673000 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:53.673000 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:53.674074 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:53.710516 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:53.711407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.712913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:53.712913 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:53.712913 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:53.712913 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:53.712913 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:53.712913 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:53.712913 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.712913 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:53.714081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.714081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:53.714081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:53.714081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:53.714081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.714081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:32:53.714081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:53.714081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:53.714081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:53.714081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:53.714081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:53.714081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:53.714630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:53.714630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:53.714630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:53.714630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:53.714630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:53.714630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:53.714630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:53.714630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:53.714630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:53.714630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:53.714630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:53.714630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:53.714630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:53.715253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.715253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:53.715461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.715461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:53.715461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:53.715461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:53.715461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:53.715461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:53.715461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.715461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:53.715461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:53.715461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.715461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:53.715461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:53.715461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.715461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:53.715461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:53.715461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.716869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:32:53.716869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:53.716869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:53.716869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:53.716869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:53.716869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:53.716869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:53.716869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:53.716869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:53.716869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:53.716869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:53.716869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:53.716869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:53.716869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:53.716869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:53.716869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:53.716869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:53.716869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:53.716869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:53.717623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:53.717623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.717623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:53.906831 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:53.907293 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:53.907499 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:53.908691 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:53.909617 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:53.909617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.910393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:53.910393 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:53.910393 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:53.910393 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:53.910393 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:53.910393 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:53.910393 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.910393 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:53.912042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.912042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:53.912042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:53.912042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:53.912042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.912042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:32:53.912042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:53.912042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:53.912042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:53.912042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:53.912042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:53.912042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:53.912583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:53.912583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:53.912583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:53.912583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:53.912583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:53.912583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:53.912583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:53.912583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:53.912583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:53.912583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:53.912583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:53.912583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:53.912583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:53.913635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.913635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:53.913862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.913862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:53.913862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:53.913862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:53.913862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:53.913862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:53.913862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.913862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:53.913862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:53.913862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.913862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:53.913862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:53.913862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.913862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:53.913862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:53.913862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.915341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:32:53.915341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:53.915341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:53.915341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:53.915341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:53.915341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:53.915341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:53.915341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:53.915341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:53.915341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:53.915341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:53.915341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:53.915341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:53.915341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:53.915341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:53.915341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:53.915341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:53.915341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:53.915341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:53.916666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:53.916666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.916666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:53.956402 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:53.956800 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:53.957003 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:53.957318 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:32:53.957720 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:53.957936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.958569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:53.958569 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:53.958569 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:53.958569 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:53.958569 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:53.958569 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:53.958569 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.958569 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:53.959794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.959794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:32:53.959794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:53.959794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:53.959794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:53.959794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:53.959794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:53.959794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:53.959794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:53.959794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:53.959794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:53.959794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:32:53.959794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:53.959794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:32:53.959794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:32:53.960549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:53.960549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:32:53.960549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:53.960549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:32:53.960549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:32:53.960549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:53.960549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:32:53.960549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.960549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:32:53.961143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.961143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:53.961143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:53.961143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:53.961143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.961143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:32:53.961143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:32:53.961143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:32:53.961143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:53.961143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:53.961143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:53.961143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:53.962720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:53.962720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:53.962720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:53.962720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:53.962720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:53.962720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:53.962720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:53.962720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:53.962720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:53.962720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:53.962720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:53.962720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:53.962720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:53.962720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:53.964733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:53.964733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:53.964733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.964733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:53.964733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.964733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:53.964733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:53.964733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:53.964733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:53.964733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:53.964733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.964733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:53.964733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:53.964733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.966112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:53.966112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:53.966112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.966112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:53.966112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:53.966112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.966112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:32:53.966112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:32:53.966112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:32:53.966112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:53.966112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:53.966112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:53.966112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:53.966112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:53.966112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:53.966112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:53.966112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:53.966112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:53.966112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:53.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:53.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:53.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:53.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:53.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:53.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:53.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:53.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:53.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:53.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:53.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:53.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:53.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:53.967734 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:53.967734 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:53.967734 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:53.967734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.967734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:53.967734 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:53.967734 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:53.967734 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:53.967734 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:53.967734 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:53.967734 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.967734 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:53.968265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.968265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:32:53.968265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:53.968265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:53.968265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:53.968265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:53.968265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:53.968265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:53.968265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:53.968265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:53.968265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:53.968265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:32:53.968265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:53.968265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:32:53.968265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:32:53.970534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:53.970534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:32:53.970534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:53.970534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:32:53.970534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:32:53.970534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:53.970534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:32:53.970534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.970534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:32:53.970534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.970534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:53.970534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:53.970534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:53.970534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.970829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:32:53.970829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:32:53.970829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:53.970829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:53.970829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:53.970829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:53.970829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:53.970829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:53.970829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:53.970829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:53.970829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:53.970829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:53.970829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:53.971081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:53.971081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:53.971081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:53.971081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:53.971081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:53.971081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:53.971081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:53.971081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:53.971081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:53.971081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.971081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:53.971081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.971321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:53.971321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:53.971321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:53.971321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:53.971321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:53.971321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.971321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:53.971321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:53.971321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.971321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:53.971321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:53.971321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.971321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:53.971321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:53.971321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.971321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:32:53.971321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:32:53.971321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:53.971321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:53.971669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:53.971669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:53.971669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:53.971669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:53.971669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:53.971669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:53.971669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:53.971669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:53.971669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:53.971669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:53.971669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:53.971669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:53.971669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:53.971669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:53.971669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:53.971669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:53.971669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:53.971669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:53.971669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.971669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:53.972230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:53.972230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:53.972230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:53.972230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:53.972230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:53.972230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:53.972368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.972368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:53.972368 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:53.972368 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:53.972368 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:53.972368 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:53.972368 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:53.972368 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.972368 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:53.972828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.972828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:53.972828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:53.972828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:53.972828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.972828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:32:53.972828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:53.972828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:53.972828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:53.972828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:53.972828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:53.972828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:53.973099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:53.973099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:53.973099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:53.973099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:53.973099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:53.973099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:53.973099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:53.973099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:53.973099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:53.973099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:53.973099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:53.973099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:53.973099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:53.973332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.973332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:53.973788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.973788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:53.973788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:53.973788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:53.973788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:53.973788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:53.973788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.973788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:53.973788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:53.973788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.973788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:53.973788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:53.973788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.973788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:53.973788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:53.973788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:53.974137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:32:53.974137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:53.974137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:53.974137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:53.974137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:53.974137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:53.974137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:53.974137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:53.974137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:53.974137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:53.974137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:53.974137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:53.974137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:53.974137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:53.974137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:53.974137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:53.974137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:53.974137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:53.974137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:53.974475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:53.974475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:53.974475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:58.672699 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:58.673344 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:58.673482 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:58.674759 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:58.710691 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:58.711128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.711866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:58.711866 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:58.711866 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:58.711866 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:58.711866 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:58.711866 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:58.711866 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.711866 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:58.712535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.712535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:58.712535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:58.712535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:58.712535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.712535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:32:58.712535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:58.712535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:58.712535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:58.712535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:58.712535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:58.712535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:58.713162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:58.713162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:58.713162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:58.713162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:58.713162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:58.713162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:58.713162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:58.713162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:58.713162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:58.713162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:58.713162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:58.713162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:58.713162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:58.713784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.713784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:58.713784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.713784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:58.713784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:58.713784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:58.713784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:58.713784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:58.713784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.713784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:58.713784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:58.713784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.713784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:58.713784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:58.713784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.714474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:58.714474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:58.714474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.714474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:32:58.714474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:58.714474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:58.714474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:58.714474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:58.714474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:58.714474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:58.714474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:58.714474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:58.714474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:58.714474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:58.714474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:58.714474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:58.714474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:58.714474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:58.714474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:58.715377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:58.715377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:58.715377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:58.715377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:58.715377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.715377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:58.907532 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:58.908193 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:58.908362 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:58.909587 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:58.910124 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:58.910317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.911262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:58.911262 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:58.911262 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:58.911262 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:58.911262 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:58.911262 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:58.911262 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.911262 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:58.912726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.912726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:58.912726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:58.912726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:58.912726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.912726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:32:58.912726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:58.912726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:58.912726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:58.912726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:58.912726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:58.912726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:58.913420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:58.913420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:58.913420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:58.913420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:58.913420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:58.913420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:58.913420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:58.913420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:58.913420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:58.913420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:58.913420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:58.913420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:58.913420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:58.914062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.914062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:58.915412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.915412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:58.915412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:58.915412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:58.915412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:58.915412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:58.915412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.915412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:58.915412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:58.915412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.915412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:58.915412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:58.915412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.915412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:58.915412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:58.915412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.916186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:32:58.916186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:58.916186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:58.916186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:58.916186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:58.916186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:58.916186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:58.916186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:58.916186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:58.916186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:58.916186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:58.916186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:58.916186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:58.916186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:58.916186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:58.916186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:58.916186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:58.916186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:58.916186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:58.917473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:58.917473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.917473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:58.957111 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:58.957445 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:58.957692 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:58.957899 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:32:58.958482 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:58.958755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.959385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:58.959385 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:58.959385 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:58.959385 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:58.959385 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:58.959385 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:58.959385 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.959385 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:58.960583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.960583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:32:58.960583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:58.960583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:58.960583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:58.960583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:58.960583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:58.960583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:58.960583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:58.960583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:58.960583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:58.960583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:32:58.960583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:58.960583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:32:58.960583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:32:58.961492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:58.961492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:32:58.961492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:58.961492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:32:58.961492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:32:58.961492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:58.961492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:32:58.961492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.961492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:32:58.962228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.962228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:58.962228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:58.962228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:58.962228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.962228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:32:58.962228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:32:58.962228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:32:58.962228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:58.962228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:58.962228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:58.962228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:58.962883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:58.962883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:58.962883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:58.962883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:58.962883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:58.962883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:58.962883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:58.962883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:58.962883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:58.962883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:58.962883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:58.962883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:58.962883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:58.962883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:58.963549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:58.963549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:58.963549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.963549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:58.963549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.963549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:58.963549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:58.963549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:58.963549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:58.963549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:58.963549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.963549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:58.963549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:58.963549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.964289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:58.964289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:58.964289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.964289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:58.964289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:58.964289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.964289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:32:58.964289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:32:58.964289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:32:58.964289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:58.964289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:58.964289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:58.964289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:58.964289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:58.964289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:58.964289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:58.964289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:58.964289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:58.964289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:58.965273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:58.965273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:58.965273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:58.965273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:58.965273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:58.965273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:58.965273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:58.965273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:58.965273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:58.965273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:58.965273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.965273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:58.965273 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:58.965273 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:58.965990 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:58.965990 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:58.965990 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:58.965990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.966225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:58.966225 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:58.966225 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:58.966225 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:58.966225 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:58.966225 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:58.966225 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.966225 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:58.966871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.966871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:32:58.966871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:58.966871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:58.966871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:58.966871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:58.966871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:58.966871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:58.966871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:58.966871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:32:58.966871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:58.966871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:32:58.966871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:58.966871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:32:58.966871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:32:58.967602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:58.967602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:32:58.967602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:58.967602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:32:58.967602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:32:58.967602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:58.967602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:32:58.967602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.967602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:32:58.967861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.967861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:58.967861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:58.967861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:58.967861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.967861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:32:58.967861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:32:58.967861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:58.967861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:58.967861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:58.967861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:58.968180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:58.968180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:58.968180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:58.968180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:58.968180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:58.968180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:58.968180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:58.968180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:58.968180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:58.968180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:58.968180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:58.968180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:58.968180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:58.968180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:58.968542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:58.968542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:58.968542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.968542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:58.968982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.968982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:58.968982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:58.968982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:58.968982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:58.968982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:58.968982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.968982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:58.968982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:58.968982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.968982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:58.968982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:58.968982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.968982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:58.968982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:58.968982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.968982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:32:58.969525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:32:58.969525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:32:58.969525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:32:58.969525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:58.969525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:58.969525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:58.969525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:58.969525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:58.969525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:58.969525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:58.969525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:58.969525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:58.969525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:58.969525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:58.969525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:58.969525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:58.969525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:58.969525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:58.969525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:58.970040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:58.970040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:58.970040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.970040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:32:58.970168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:32:58.970328 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:32:58.970438 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:32:58.970590 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:32:58.970935 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:58.971163 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:32:58.971316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.971809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:32:58.971809 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:58.971809 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:32:58.971809 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:32:58.971809 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:58.971809 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:32:58.971809 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.971809 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:32:58.972662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.972662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:32:58.972662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:58.972662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:32:58.972662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.972662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:32:58.972662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:58.972662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:58.972662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:58.972662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:58.972662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:58.972662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:58.973088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:32:58.973088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:58.973088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:32:58.973088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:58.973088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:32:58.973088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:32:58.973088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:58.973088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:32:58.973088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:58.973088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:32:58.973088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:32:58.973088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:58.973088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:32:58.973470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.973470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:32:58.973672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.973672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:32:58.973672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:32:58.973672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:32:58.973672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:32:58.973672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:32:58.973672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.973672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:32:58.973672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:32:58.973672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.973672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:32:58.973672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:32:58.973672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.973672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:32:58.973672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:32:58.973672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:32:58.974161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:32:58.974161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:32:58.974161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:58.974161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:58.974161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:32:58.974161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:32:58.974161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:32:58.974161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:32:58.974161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:32:58.974161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:32:58.974161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:32:58.974161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:32:58.974161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:32:58.974161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:32:58.974161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:32:58.974161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:32:58.974161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:32:58.974161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:32:58.974161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:32:58.974723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:32:58.974723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:32:58.974723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:03.673233 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:03.673934 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:03.674061 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:03.675048 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:03.715901 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:03.716477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.717091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:03.717091 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:03.717091 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:03.717091 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:03.717091 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:03.717091 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:03.717091 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.717091 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:03.718011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.718011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:03.718011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:03.718011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:03.718011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.718011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:33:03.718011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:03.718011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:03.718011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:03.718011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:03.718011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:03.718011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:03.718404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:03.718404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:03.718404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:03.718404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:03.718404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:03.718404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:03.718404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:03.718404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:03.718404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:03.718404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:03.718404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:03.718404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:03.718404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:03.718814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.718814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:03.718814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.718814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:03.718814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:03.718814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:03.718814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:03.718814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:03.718814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.718814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:03.718814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:03.718814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.718814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:03.718814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:03.718814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.719310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:03.719310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:03.719310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.719310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:33:03.719310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:03.719310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:03.719310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:03.719310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:03.719310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:03.719310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:03.719310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:03.719310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:03.719310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:03.719310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:03.719310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:03.719310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:03.719310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:03.719310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:03.719310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:03.719876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:03.719876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:03.719876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:03.719876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:03.719876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.719876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:03.908366 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:03.909165 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:03.909356 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:03.910649 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:03.911037 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:03.911369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.912176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:03.912176 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:03.912176 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:03.912176 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:03.912176 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:03.912176 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:03.912176 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.912176 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:03.914327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.914327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:03.914327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:03.914327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:03.914327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.914327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:33:03.914327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:03.914327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:03.914327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:03.914327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:03.914327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:03.914327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:03.914687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:03.914687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:03.914687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:03.914687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:03.914687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:03.914687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:03.914687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:03.914687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:03.914687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:03.914687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:03.914687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:03.914687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:03.914687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:03.915155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.915155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:03.915961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.915961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:03.915961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:03.915961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:03.915961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:03.915961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:03.915961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.915961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:03.915961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:03.915961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.915961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:03.915961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:03.915961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.915961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:03.915961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:03.915961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.916405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:33:03.916405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:03.916405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:03.916405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:03.916405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:03.916405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:03.916405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:03.916405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:03.916405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:03.916405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:03.916405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:03.916405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:03.916405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:03.916405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:03.916405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:03.916405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:03.916405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:03.916405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:03.916405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:03.917138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:03.917138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.917138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:03.957706 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:03.958328 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:03.958582 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:03.958813 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:33:03.959568 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:03.959863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.960652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:03.960652 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:03.960652 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:03.960652 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:03.960652 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:03.960652 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:03.960652 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.960652 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:03.962151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.962151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:33:03.962151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:03.962151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:03.962151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:03.962151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:03.962151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:03.962151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:03.962151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:03.962151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:03.962151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:03.962151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:33:03.962151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:03.962151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:33:03.962151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:33:03.962864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:03.962864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:33:03.962864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:03.962864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:33:03.962864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:33:03.962864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:03.962864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:33:03.962864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.962864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:33:03.963700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.963700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:03.963700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:03.963700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:03.963700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.963700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:33:03.963700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:33:03.963700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:33:03.963700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:03.963700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:03.963700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:03.963700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:03.964104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:03.964104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:03.964104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:03.964104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:03.964104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:03.964104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:03.964104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:03.964104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:03.964104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:03.964104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:03.964104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:03.964104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:03.964104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:03.964104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:03.964461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:03.964461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:03.964461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.964461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:03.965497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.965497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:03.965497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:03.965497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:03.965497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:03.965497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:03.965497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.965497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:03.965497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:03.965497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.965497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:03.965497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:03.965497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.965497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:03.965497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:03.965497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.965954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:33:03.965954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:33:03.965954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:33:03.965954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:03.965954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:03.965954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:03.965954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:03.965954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:03.965954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:03.965954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:03.965954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:03.965954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:03.965954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:03.965954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:03.965954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:03.965954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:03.965954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:03.965954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:03.966430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:03.966430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:03.966430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:03.966430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:03.966430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:03.966430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.966430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:03.966430 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:03.966644 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:03.966644 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:03.966833 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:03.967361 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:03.967618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.968523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:03.968523 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:03.968523 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:03.968523 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:03.968523 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:03.968523 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:03.968523 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.968523 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:03.969858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.969858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:33:03.969858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:03.969858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:03.969858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:03.969858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:03.969858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:03.969858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:03.969858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:03.969858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:03.969858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:03.969858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:33:03.969858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:03.969858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:33:03.969858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:33:03.970425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:03.970425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:33:03.970425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:03.970425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:33:03.970425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:33:03.970425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:03.970425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:33:03.970425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.970425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:33:03.970425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.970425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:03.970425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:03.970425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:03.970425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.970914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:33:03.970914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:33:03.970914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:03.970914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:03.970914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:03.970914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:03.970914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:03.970914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:03.970914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:03.970914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:03.970914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:03.970914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:03.970914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:03.971347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:03.971347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:03.971347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:03.971347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:03.971347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:03.971347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:03.971347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:03.971347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:03.971347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:03.971347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.971347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:03.971659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.971659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:03.971659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:03.971659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:03.971659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:03.971659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:03.971659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.971659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:03.971659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:03.971659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.971659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:03.971659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:03.971659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.971659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:03.971659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:03.971659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.971659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:33:03.972214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:33:03.972214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:03.972214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:03.972214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:03.972214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:03.972214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:03.972214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:03.972214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:03.972214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:03.972214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:03.972214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:03.972214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:03.972214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:03.972214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:03.972214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:03.972214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:03.972214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:03.972214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:03.972214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:03.973113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:03.973113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:03.973113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.973113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:03.973113 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:03.973113 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:03.973113 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:03.973113 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:03.974276 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:03.974830 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:03.975097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.976178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:03.976178 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:03.976178 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:03.976178 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:03.976178 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:03.976178 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:03.976178 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.976178 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:03.977051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.977051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:03.977051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:03.977051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:03.977051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.977051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:33:03.977051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:03.977051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:03.977051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:03.977051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:03.977051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:03.977051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:03.977547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:03.977547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:03.977547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:03.977547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:03.977547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:03.977547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:03.977547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:03.977547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:03.977547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:03.977547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:03.977547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:03.977547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:03.977547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:03.978068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.978068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:03.978068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.978068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:03.978068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:03.978068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:03.978068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:03.978068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:03.978068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.978068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:03.978068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:03.978068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.978068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:03.978068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:03.978068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.978564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:03.978564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:03.978564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:03.978564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:33:03.978564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:03.978564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:03.978564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:03.978564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:03.978564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:03.978564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:03.978564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:03.978564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:03.978564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:03.978564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:03.978564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:03.978564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:03.978564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:03.978564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:03.978564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:03.979208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:03.979208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:03.979208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:03.979208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:03.979208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:03.979208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:08.673968 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:08.674792 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:08.674980 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:08.676704 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:08.679317 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=240 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:33:08.712182 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:08.713407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.713540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:08.713540 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:08.713540 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:08.713540 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:08.713540 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:08.713540 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:08.713540 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.713540 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:08.714308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.714308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:08.714308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:08.714308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:08.714308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.714308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:33:08.714308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:08.714308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:08.714308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:08.714308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:08.714308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:08.714308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:08.714755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:08.714755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:08.714755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:08.714755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:08.714755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:08.714755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:08.714755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:08.714755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:08.714755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:08.714755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:08.714755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:08.714755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:08.714755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:08.715205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.715205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:08.715205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.715205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:08.715205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:08.715205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:08.715205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:08.715205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:08.715205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.715205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:08.715205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:08.715205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.715205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:08.715205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:08.715205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.715751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:08.715751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:08.715751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.715751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:33:08.715751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:08.715751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:08.715751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:08.715751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:08.715751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:08.715751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:08.715751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:08.715751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:08.715751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:08.715751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:08.715751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:08.715751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:08.715751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:08.715751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:08.715751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:08.716381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:08.716381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:08.716381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:08.716381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:08.716381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.716381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:08.909020 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:08.909586 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:08.909844 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:08.912121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:08.912121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:08.912121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.913358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:08.913358 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:08.913358 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:08.913358 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:08.913358 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:08.913358 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:08.913358 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.913358 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:08.914080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.914080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:08.914080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:08.914080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:08.914080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.914080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:33:08.914080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:08.914080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:08.914080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:08.914080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:08.914080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:08.914080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:08.914653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:08.914653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:08.914653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:08.914653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:08.914653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:08.914653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:08.914653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:08.914653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:08.914653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:08.914653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:08.914653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:08.914653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:08.914653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:08.915566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.915566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:08.915566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.915566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:08.915566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:08.915566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:08.915566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:08.915566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:08.915566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.915566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:08.915566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:08.915566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.915566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:08.915566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:08.915566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.916288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:08.916288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:08.916288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.916288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:33:08.916288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:08.916288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:08.916288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:08.916288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:08.916288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:08.916288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:08.916288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:08.916288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:08.916288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:08.916288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:08.916288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:08.916288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:08.916288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:08.916288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:08.916288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:08.917121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:08.917121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:08.917121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:08.917121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:08.917121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.917121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:08.958437 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:08.958944 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:08.959124 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:08.959325 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:33:08.960778 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:08.960778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:08.964583 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:08.964583 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:08.964583 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:08.964583 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:08.964583 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:08.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:08.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:33:08.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:08.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:08.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:08.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:08.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:08.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:08.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:08.964583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:08.965525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:08.965525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:33:08.965525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:08.965525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:33:08.965525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:33:08.965525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:08.965525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:33:08.965525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:08.965525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:33:08.965525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:33:08.965525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:08.965525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:33:08.965525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.965525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:33:08.968023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.968023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:08.968023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:08.968023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:08.968023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.968023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:33:08.968023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:33:08.968023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:33:08.968023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:08.968023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:08.968023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:08.968023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:08.968853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:08.968853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:08.968853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:08.968853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:08.968853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:08.968853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:08.968853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:08.968853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:08.968853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:08.968853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:08.968853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:08.968853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:08.968853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:08.968853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:08.969210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:08.969210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:08.969210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.969210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:08.970042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.970042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:08.970042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:08.970042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:08.970042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:08.970042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:08.970042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.970042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:08.970042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:08.970042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.970042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:08.970042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:08.970042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.970042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:08.970042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:08.970042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.971115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:33:08.971115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:33:08.971115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:33:08.971115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:08.971115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:08.971115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:08.971115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:08.971115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:08.971115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:08.971115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:08.971115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:08.971115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:08.971115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:08.971115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:08.971115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:08.971115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:08.971115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:08.971115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:08.971578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:08.971578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:08.971578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:08.971578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:08.971578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:08.971578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.971578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:08.971578 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:08.971578 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:08.971578 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:08.972204 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:08.972204 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:08.972204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.972300 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:08.973926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:08.973926 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:08.973926 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:08.973926 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:08.973926 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:08.973926 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:08.973926 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.973926 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:08.975115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.975115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:33:08.975115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:08.975115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:08.975115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:08.975115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:08.975115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:08.975115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:08.975115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:08.975115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:08.975115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:08.975115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:33:08.975115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:08.975115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:33:08.975115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:33:08.975515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:08.975515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:33:08.975515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:08.975515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:33:08.975515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:33:08.975515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:08.975515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:33:08.975515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.975515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:33:08.976490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.976490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:08.976490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:08.976490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:08.976490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.976490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:33:08.976490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:33:08.976490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:08.976490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:08.976490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:08.976490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:08.976924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:08.976924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:08.976924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:08.976924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:08.976924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:08.976924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:08.976924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:08.976924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:08.976924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:08.976924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:08.976924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:08.976924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:08.976924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:08.976924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:08.977254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:08.977254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:08.977254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.977254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:08.978263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.978263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:08.978263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:08.978263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:08.978263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:08.978263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:08.978263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.978263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:08.978263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:08.978263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.978263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:08.978263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:08.978263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.978263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:08.978263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:08.978263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.978263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:33:08.978767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:33:08.978767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:08.978767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:08.978767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:08.978767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:08.978767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:08.978767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:08.978767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:08.978767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:08.978767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:08.978767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:08.978767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:08.978767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:08.978767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:08.978767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:08.978767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:08.978767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:08.978767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:08.978767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:08.979208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:08.979208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:08.979208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.979208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:08.979208 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:08.979208 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:08.979208 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:08.979388 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:08.979500 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:08.979614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.980065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:08.980065 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:08.980065 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:08.980065 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:08.980065 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:08.980065 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:08.980065 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.980065 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:08.980897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.980897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:08.980897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:08.980897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:08.980897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.980897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:33:08.980897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:08.980897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:08.980897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:08.980897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:08.980897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:08.980897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:08.981209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:08.981209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:08.981209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:08.981209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:08.981209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:08.981209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:08.981209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:08.981209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:08.981209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:08.981209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:08.981209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:08.981209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:08.981209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:08.981519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.981519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:08.981954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.981954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:08.981954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:08.981954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:08.981954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:08.981954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:08.981954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.981954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:08.981954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:08.981954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.981954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:08.981954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:08.981954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.981954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:08.981954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:08.981954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:08.982370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:33:08.982370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:08.982370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:08.982370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:08.982370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:08.982370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:08.982370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:08.982370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:08.982370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:08.982370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:08.982370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:08.982370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:08.982370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:08.982370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:08.982370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:08.982370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:08.982370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:08.982370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:08.982370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:08.982849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:08.982849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:08.982849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:13.674742 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:13.675459 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:13.675530 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:13.676781 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:13.713451 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:13.713617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.714156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:13.714156 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:13.714156 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:13.714156 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:13.714156 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:13.714156 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:13.714156 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.714156 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:13.714805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.714805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:13.714805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:13.714805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:13.714805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.714805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:33:13.714805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:13.714805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:13.714805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:13.714805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:13.714805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:13.714805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:13.715135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:13.715135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:13.715135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:13.715135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:13.715135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:13.715135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:13.715135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:13.715135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:13.715135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:13.715135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:13.715135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:13.715135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:13.715135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:13.715746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.715746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:13.715746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.715746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:13.715746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:13.715746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:13.715746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:13.715746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:13.715746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.715746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:13.715746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:13.715746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.715746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:13.715746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:13.715746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.716592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:13.716592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:13.716592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.716592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:33:13.716592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:13.716592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:13.716592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:13.716592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:13.716592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:13.716592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:13.716592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:13.716592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:13.716592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:13.716592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:13.716592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:13.716592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:13.716592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:13.716592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:13.716592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:13.717534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:13.717534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:13.717534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:13.717534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:13.717534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.717534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:13.909663 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:13.910220 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:13.910342 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:13.911006 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:13.911188 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:13.911336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.911899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:13.911899 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:13.911899 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:13.911899 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:13.911899 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:13.911899 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:13.911899 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.911899 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:13.912930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.912930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:13.912930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:13.912930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:13.912930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.912930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:33:13.912930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:13.912930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:13.912930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:13.912930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:13.912930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:13.912930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:13.913511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:13.913511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:13.913511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:13.913511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:13.913511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:13.913511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:13.913511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:13.913511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:13.913511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:13.913511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:13.913511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:13.913511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:13.913511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:13.914115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.914115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:13.914539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.914539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:13.914539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:13.914539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:13.914539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:13.914539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:13.914539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.914539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:13.914539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:13.914539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.914539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:13.914539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:13.914539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.914539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:13.914539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:13.914539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.915274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:33:13.915274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:13.915274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:13.915274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:13.915274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:13.915274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:13.915274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:13.915274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:13.915274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:13.915274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:13.915274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:13.915274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:13.915274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:13.915274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:13.915274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:13.915274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:13.915274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:13.915274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:13.915274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:13.916090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:13.916090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.916090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:13.958994 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:13.959387 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:13.959668 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:13.959668 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:33:13.960007 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:13.960254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.960953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:13.960953 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:13.960953 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:13.960953 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:13.960953 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:13.960953 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:13.960953 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.960953 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:13.962041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.962041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:33:13.962041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:13.962041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:13.962041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:13.962041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:13.962041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:13.962041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:13.962041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:13.962041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:13.962041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:13.962041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:33:13.962041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:13.962041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:33:13.962041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:33:13.962705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:13.962705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:33:13.962705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:13.962705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:33:13.962705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:33:13.962705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:13.962705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:33:13.962705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.962705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:33:13.963178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.963178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:13.963178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:13.963178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:13.963178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.963178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:33:13.963178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:33:13.963178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:33:13.963178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:13.963178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:13.963178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:13.963178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:13.963706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:13.963706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:13.963706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:13.963706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:13.963706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:13.963706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:13.963706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:13.963706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:13.963706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:13.963706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:13.963706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:13.963706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:13.963706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:13.963706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:13.964585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:13.964585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:13.964585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.964585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:13.964585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.964585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:13.964585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:13.964585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:13.964585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:13.964585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:13.964585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.964585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:13.964585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:13.964585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.965343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:13.965343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:13.965343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.965343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:13.965343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:13.965343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.965343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:33:13.965343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:33:13.965343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:33:13.965343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:13.965343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:13.965343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:13.965343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:13.965343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:13.965343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:13.965343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:13.965343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:13.965343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:13.965343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:13.966307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:13.966307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:13.966307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:13.966307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:13.966307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:13.966307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:13.966307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:13.966307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:13.966307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:13.966307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:13.966307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.966307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:13.970652 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:13.970977 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:13.971123 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:13.971268 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:13.971515 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:13.971775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.972356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:13.972356 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:13.972356 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:13.972356 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:13.972356 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:13.972356 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:13.972356 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.972356 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:13.973303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.973303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:33:13.973303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:13.973303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:13.973303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:13.973303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:13.973303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:13.973303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:13.973303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:13.973303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:13.973303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:13.973303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:33:13.973303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:13.973303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:33:13.973303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:33:13.973728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:13.973728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:33:13.973728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:13.973728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:33:13.973728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:33:13.973728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:13.973728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:33:13.973728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.973728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:33:13.974326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.974326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:13.974326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:13.974326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:13.974326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.974326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:33:13.974326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:33:13.974326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:13.974326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:13.974326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:13.974326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:13.974697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:13.974697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:13.974697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:13.974697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:13.974697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:13.974697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:13.974697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:13.974697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:13.974697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:13.974697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:13.974697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:13.974697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:13.974697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:13.974697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:13.975128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:13.975128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:13.975128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.975128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:13.975441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.975441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:13.975441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:13.975441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:13.975441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:13.975441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:13.975441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.975441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:13.975441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:13.975441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.975441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:13.975441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:13.975441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.975441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:13.975441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:13.975441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.975441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:33:13.975928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:33:13.975928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:13.975928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:13.975928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:13.975928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:13.975928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:13.975928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:13.975928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:13.975928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:13.975928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:13.975928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:13.975928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:13.975928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:13.975928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:13.975928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:13.975928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:13.975928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:13.975928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:13.975928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:13.976474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:13.976474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:13.976474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.976474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:13.978652 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:13.978950 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:13.979067 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:13.979213 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:13.979440 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:13.979654 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:13.979814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.980268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:13.980268 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:13.980268 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:13.980268 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:13.980268 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:13.980268 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:13.980268 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.980268 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:13.981294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.981294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:13.981294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:13.981294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:13.981294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.981294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:33:13.981294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:13.981294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:13.981294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:13.981294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:13.981294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:13.981294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:13.981634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:13.981634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:13.981634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:13.981634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:13.981634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:13.981634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:13.981634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:13.981634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:13.981634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:13.981634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:13.981634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:13.981634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:13.981634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:13.982004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.982004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:13.982331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.982331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:13.982331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:13.982331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:13.982331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:13.982331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:13.982331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.982331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:13.982331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:13.982331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.982331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:13.982331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:13.982331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.982331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:13.982331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:13.982331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:13.982781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:33:13.982781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:13.982781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:13.982781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:13.982781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:13.982781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:13.982781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:13.982781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:13.982781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:13.982781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:13.982781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:13.982781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:13.982781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:13.982781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:13.982781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:13.982781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:13.982781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:13.982781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:13.982781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:13.983378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:13.983378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:13.983378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:18.675342 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:18.676274 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:18.676274 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:18.677316 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:18.709042 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:18.709532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.710447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:18.710447 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:18.710447 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:18.710447 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:18.710447 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:18.710447 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:18.710447 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.710447 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:18.711426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.711426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:18.711426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:18.711426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:18.711426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.711426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:33:18.711426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:18.711426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:18.711426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:18.711426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:18.711426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:18.711426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:18.712125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:18.712125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:18.712125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:18.712125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:18.712125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:18.712125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:18.712125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:18.712125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:18.712125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:18.712125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:18.712125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:18.712125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:18.712125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:18.712805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.712805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:18.712805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.712805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:18.712805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:18.712805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:18.712805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:18.712805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:18.712805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.712805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:18.712805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:18.712805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.712805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:18.712805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:18.712805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.713570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:18.713570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:18.713570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.713570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:33:18.713570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:18.713570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:18.713570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:18.713570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:18.713570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:18.713570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:18.713570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:18.713570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:18.713570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:18.713570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:18.713570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:18.713570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:18.713570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:18.713570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:18.713570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:18.714496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:18.714496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:18.714496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:18.714496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:18.714496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.714496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:18.910387 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:18.911085 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:18.911324 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:18.912536 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:18.912958 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:18.913255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.914249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:18.914249 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:18.914249 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:18.914249 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:18.914249 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:18.914249 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:18.914249 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.914249 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:18.915829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.915829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:18.915829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:18.915829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:18.915829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.915829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:33:18.915829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:18.915829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:18.915829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:18.915829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:18.915829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:18.915829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:18.916207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:18.916207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:18.916207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:18.916207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:18.916207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:18.916207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:18.916207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:18.916207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:18.916207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:18.916207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:18.916207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:18.916207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:18.916207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:18.916526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.916526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:18.918339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.918339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:18.918339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:18.918339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:18.918339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:18.918339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:18.918339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.918339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:18.918339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:18.918339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.918339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:18.918339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:18.918339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.918339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:18.918339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:18.918339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.918841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:33:18.918841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:18.918841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:18.918841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:18.918841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:18.918841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:18.918841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:18.918841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:18.918841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:18.918841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:18.918841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:18.918841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:18.918841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:18.918841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:18.918841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:18.918841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:18.918841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:18.918841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:18.918841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:18.919332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:18.919332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.919332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:18.959393 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:18.959896 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:18.960081 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:18.960265 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:33:18.961083 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:18.961454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.962472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:18.962472 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:18.962472 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:18.962472 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:18.962472 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:18.962472 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:18.962472 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.962472 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:18.964142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.964142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:33:18.964142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:18.964142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:18.964142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:18.964142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:18.964142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:18.964142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:18.964142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:18.964142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:18.964142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:18.964142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:33:18.964142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:18.964142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:33:18.964142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:33:18.965060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:18.965060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:33:18.965060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:18.965060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:33:18.965060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:33:18.965060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:18.965060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:33:18.965060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.965060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:33:18.966042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.966042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:18.966042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:18.966042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:18.966042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.966042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:33:18.966042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:33:18.966042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:33:18.966042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:18.966042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:18.966042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:18.966042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:18.966727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:18.966727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:18.966727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:18.966727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:18.966727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:18.966727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:18.966727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:18.966727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:18.966727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:18.966727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:18.966727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:18.966727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:18.966727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:18.966727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:18.967458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:18.967458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:18.967458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.967458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:18.968120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.968120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:18.968120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:18.968120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:18.968120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:18.968120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:18.968120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.968120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:18.968120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:18.968120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.968120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:18.968120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:18.968120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.968120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:18.968120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:18.968120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.969333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:33:18.969333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:33:18.969333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:33:18.969333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:18.969333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:18.969333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:18.969333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:18.969333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:18.969333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:18.969333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:18.969333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:18.969333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:18.969333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:18.969333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:18.969333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:18.969333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:18.969333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:18.969333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:18.970196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:18.970196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:18.970196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:18.970196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:18.970196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:18.970196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.970196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:18.971437 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:18.971549 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:18.971606 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:18.971807 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:18.972553 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:18.972886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.975605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:18.975605 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:18.975605 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:18.975605 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:18.975605 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:18.975605 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:18.975605 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.975605 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:18.975605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.975605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:33:18.975605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:18.975605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:18.975605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:18.975605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:18.975605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:18.975605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:18.975605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:18.975605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:18.976157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:18.976157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:33:18.976157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:18.976157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:33:18.976157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:33:18.976157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:18.976157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:33:18.976157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:18.976157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:33:18.976157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:33:18.976157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:18.976157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:33:18.976157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.976157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:33:18.976157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.976633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:18.976633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:18.976633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:18.976633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.976633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:33:18.976633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:33:18.976633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:18.976633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:18.976633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:18.976633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:18.976633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:18.976633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:18.976633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:18.977047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:18.977047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:18.977047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:18.977047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:18.977047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:18.977047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:18.977047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:18.977047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:18.977047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:18.977047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:18.977047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:18.977047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:18.977047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:18.977447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.977447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:18.977447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.977447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:18.977447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:18.977447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:18.977447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:18.977447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:18.977447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.977447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:18.977447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:18.977447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.977447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:18.977447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:18.977447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.977920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:18.977920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:18.977920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.977920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:33:18.977920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:33:18.977920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:18.977920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:18.977920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:18.977920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:18.977920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:18.977920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:18.977920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:18.977920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:18.977920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:18.977920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:18.977920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:18.977920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:18.977920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:18.977920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:18.978491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:18.978491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:18.978491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:18.978491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:18.978491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:18.978491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:18.978491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.978491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:18.979085 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:18.979425 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:18.979537 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:18.979719 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:18.980008 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:18.980178 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:18.980292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.980746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:18.980746 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:18.980746 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:18.980746 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:18.980746 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:18.980746 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:18.980746 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.980746 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:18.981749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.981749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:18.981749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:18.981749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:18.981749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.981749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:33:18.981749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:18.981749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:18.981749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:18.981749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:18.981749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:18.981749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:18.982044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:18.982044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:18.982044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:18.982044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:18.982044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:18.982044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:18.982044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:18.982044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:18.982044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:18.982044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:18.982044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:18.982044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:18.982044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:18.982275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.982275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:18.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:18.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:18.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:18.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:18.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:18.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:18.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:18.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:18.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:18.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:18.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:18.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:18.983201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:33:18.983201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:18.983201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:18.983201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:18.983201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:18.983201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:18.983201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:18.983201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:18.983201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:18.983201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:18.983201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:18.983201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:18.983201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:18.983201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:18.983201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:18.983201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:18.983201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:18.983201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:18.983201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:18.983537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:18.983537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:18.983537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:23.676434 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:23.677832 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:23.678039 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:23.679451 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:23.715107 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:23.715476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.716252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:23.716252 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:23.716252 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:23.716252 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:23.716252 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:23.716252 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:23.716252 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.716252 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:23.717252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.717252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:23.717252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:23.717252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:23.717252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.717252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:33:23.717252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:23.717252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:23.717252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:23.717252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:23.717252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:23.717252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:23.717649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:23.717649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:23.717649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:23.717649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:23.717649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:23.717649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:23.717649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:23.717649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:23.717649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:23.717649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:23.717649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:23.717649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:23.717649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:23.718012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.718012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:23.718415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.718415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:23.718415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:23.718415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:23.718415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:23.718415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:23.718415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.718415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:23.718415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:23.718415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.718415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:23.718415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:23.718415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.718415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:23.718415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:23.718415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.718922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:33:23.718922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:23.718922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:23.718922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:23.718922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:23.718922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:23.718922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:23.718922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:23.718922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:23.718922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:23.718922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:23.718922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:23.718922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:23.718922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:23.718922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:23.718922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:23.718922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:23.718922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:23.718922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:23.719648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:23.719648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.719648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:23.911342 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:23.911809 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:23.912016 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:23.913413 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:23.913413 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:23.915080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.915080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:23.915080 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:23.915080 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:23.915080 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:23.915080 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:23.915080 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:23.915080 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.915080 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:23.915956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.915956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:23.915956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:23.915956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:23.915956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.915956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:33:23.915956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:23.915956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:23.915956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:23.915956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:23.915956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:23.915956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:23.916557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:23.916557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:23.916557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:23.916557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:23.916557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:23.916557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:23.916557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:23.916557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:23.916557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:23.916557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:23.916557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:23.916557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:23.916557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:23.918320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.918320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:23.918320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.918320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:23.918320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:23.918320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:23.918320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:23.918320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:23.918320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.918320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:23.918320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:23.918320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.918320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:23.918320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:23.918320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.920304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:23.920304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:23.920304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.920304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:33:23.920304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:23.920304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:23.920304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:23.920304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:23.920304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:23.920304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:23.920304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:23.920304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:23.920304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:23.920304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:23.920304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:23.920304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:23.920304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:23.920304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:23.920304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:23.921944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:23.921944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:23.921944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:23.921944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:23.921944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.921944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:23.959996 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:23.960434 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:23.960629 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:23.960903 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:33:23.961315 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:23.961503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.962210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:23.962210 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:23.962210 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:23.962210 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:23.962210 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:23.962210 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:23.962210 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.962210 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:23.963219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.963219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:33:23.963219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:23.963219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:23.963219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:23.963219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:23.963219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:23.963219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:23.963219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:23.963219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:23.963219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:23.963219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:33:23.963219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:23.963219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:33:23.963219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:33:23.963739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:23.963739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:33:23.963739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:23.963739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:33:23.963739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:33:23.963739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:23.963739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:33:23.963739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.963739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:33:23.964071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.964071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:23.964071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:23.964071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:23.964071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.964071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:33:23.964071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:33:23.964071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:33:23.964071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:23.964071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:23.964071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:23.964071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:23.964453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:23.964453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:23.964453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:23.964453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:23.964453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:23.964453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:23.964453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:23.964453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:23.964453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:23.964453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:23.964453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:23.964453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:23.964453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:23.964453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:23.964869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:23.964869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:23.964869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.964869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:23.965157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.965157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:23.965157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:23.965157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:23.965157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:23.965157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:23.965157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.965157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:23.965157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:23.965157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.965157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:23.965157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:23.965157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.965157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:23.965157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:23.965157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.965635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:33:23.965635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:33:23.965635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:33:23.965635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:23.965635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:23.965635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:23.965635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:23.965635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:23.965635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:23.965635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:23.965635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:23.965635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:23.965635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:23.965635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:23.965635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:23.965635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:23.965635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:23.965635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:23.966431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:23.966431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:23.966431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:23.966431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:23.966431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:23.966431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.966431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:23.971654 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:23.971952 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:23.972070 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:23.972197 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:23.972497 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:23.972731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.973332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:23.973332 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:23.973332 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:23.973332 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:23.973332 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:23.973332 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:23.973332 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.973332 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:23.974143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.974143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:33:23.974143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:23.974143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:23.974143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:23.974143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:23.974143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:23.974143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:23.974143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:23.974143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:23.974143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:23.974143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:33:23.974143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:23.974143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:33:23.974143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:33:23.974644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:23.974644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:33:23.974644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:23.974644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:33:23.974644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:33:23.974644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:23.974644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:33:23.974644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.974644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:33:23.975186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.975186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:23.975186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:23.975186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:23.975186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.975186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:33:23.975186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:33:23.975186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:23.975186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:23.975186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:23.975186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:23.975950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:23.975950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:23.975950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:23.975950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:23.975950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:23.975950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:23.975950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:23.975950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:23.975950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:23.975950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:23.975950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:23.975950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:23.975950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:23.975950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:23.976752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:23.976752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:23.976752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.976752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:23.976752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.976752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:23.976752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:23.976752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:23.976752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:23.976752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:23.976752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.976752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:23.976752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:23.976752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.977129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:23.977129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:23.977129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.977129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:23.977129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:23.977129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.977129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:33:23.977129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:33:23.977129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:23.977129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:23.977129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:23.977129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:23.977129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:23.977129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:23.977129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:23.977129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:23.977129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:23.977129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:23.977129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:23.977617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:23.977617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:23.977617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:23.977617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:23.977617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:23.977617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:23.977617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:23.977617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:23.977617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:23.977617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.977617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:23.979701 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:23.980759 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:23.981081 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:23.981353 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:23.981782 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:23.982064 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:23.982251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.984073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:23.984073 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:23.984073 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:23.984073 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:23.984073 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:23.984073 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:23.984073 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.984073 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:23.985331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.985331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:23.985331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:23.985331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:23.985331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.985331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:33:23.985331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:23.985331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:23.985331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:23.985331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:23.985331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:23.985331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:23.986780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:23.986780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:23.986780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:23.986780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:23.986780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:23.986780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:23.986780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:23.986780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:23.986780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:23.986780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:23.986780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:23.986780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:23.986780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:23.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:23.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:23.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:23.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:23.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:23.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:23.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:23.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:23.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:23.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:23.987457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.988418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:23.988418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:23.988418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:23.988418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:33:23.988418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:23.988418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:23.988418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:23.988418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:23.988418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:23.988418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:23.988418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:23.988418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:23.988418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:23.988418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:23.988418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:23.988418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:23.988418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:23.988418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:23.988418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:23.989520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:23.989520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:23.989520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:23.989520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:23.989520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:23.989520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:28.677805 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:28.678379 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:28.678379 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:28.679131 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:28.716942 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:28.717451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.718367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:28.718367 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:28.718367 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:28.718367 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:28.718367 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:28.718367 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:28.718367 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.718367 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:28.719243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.719243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:28.719243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:28.719243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:28.719243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.719243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:33:28.719243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:28.719243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:28.719243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:28.719243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:28.719243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:28.719243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:28.719717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:28.719717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:28.719717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:28.719717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:28.719717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:28.719717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:28.719717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:28.719717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:28.719717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:28.719717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:28.719717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:28.719717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:28.719717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:28.720253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.720253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:28.720371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.720371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:28.720371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:28.720371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:28.720371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:28.720371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:28.720371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.720371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:28.720371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:28.720371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.720371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:28.720371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:28.720371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.720371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:28.720371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:28.720371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.721150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:33:28.721150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:28.721150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:28.721150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:28.721150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:28.721150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:28.721150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:28.721150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:28.721150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:28.721150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:28.721150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:28.721150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:28.721150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:28.721150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:28.721150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:28.721150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:28.721150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:28.721150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:28.721150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:28.722082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:28.722082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.722082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:28.911944 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:28.913050 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:28.913252 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:28.914378 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:28.914758 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:28.915006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.915891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:28.915891 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:28.915891 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:28.915891 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:28.915891 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:28.915891 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:28.915891 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.915891 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:28.917467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.917467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:28.917467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:28.917467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:28.917467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.917467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:33:28.917467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:28.917467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:28.917467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:28.917467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:28.917467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:28.917467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:28.918110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:28.918110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:28.918110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:28.918110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:28.918110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:28.918110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:28.918110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:28.918110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:28.918110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:28.918110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:28.918110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:28.918110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:28.918110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:28.918700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.918700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:28.919286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.919286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:28.919286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:28.919286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:28.919286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:28.919286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:28.919286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.919286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:28.919286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:28.919286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.919286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:28.919286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:28.919286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.919286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:28.919286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:28.919286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.920169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:33:28.920169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:28.920169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:28.920169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:28.920169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:28.920169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:28.920169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:28.920169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:28.920169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:28.920169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:28.920169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:28.920169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:28.920169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:28.920169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:28.920169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:28.920169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:28.920169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:28.920169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:28.920169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:28.921146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:28.921146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.921146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:28.960428 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:28.961129 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:28.961371 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:28.961646 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:33:28.962361 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:28.962695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.963537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:28.963537 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:28.963537 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:28.963537 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:28.963537 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:28.963537 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:28.963537 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.963537 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:28.964975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.964975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:33:28.964975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:28.964975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:28.964975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:28.964975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:28.964975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:28.964975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:28.964975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:28.964975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:28.964975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:28.964975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:33:28.964975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:28.964975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:33:28.964975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:33:28.965769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:28.965769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:33:28.965769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:28.965769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:33:28.965769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:33:28.965769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:28.965769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:33:28.965769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.965769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:33:28.966291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.966291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:28.966291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:28.966291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:28.966291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.966291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:33:28.966291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:33:28.966291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:33:28.966291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:28.966291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:28.966291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:28.966291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:28.966872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:28.966872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:28.966872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:28.966872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:28.966872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:28.966872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:28.966872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:28.966872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:28.966872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:28.966872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:28.966872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:28.966872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:28.966872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:28.966872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:28.967490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:28.967490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:28.967490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.967490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:28.967966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.967966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:28.967966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:28.967966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:28.967966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:28.967966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:28.967966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.967966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:28.967966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:28.967966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.967966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:28.967966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:28.967966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.967966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:28.967966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:28.967966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.968762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:33:28.968762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:33:28.968762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:33:28.968762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:28.968762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:28.968762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:28.968762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:28.968762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:28.968762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:28.968762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:28.968762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:28.968762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:28.968762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:28.968762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:28.968762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:28.968762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:28.968762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:28.968762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:28.969607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:28.969607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:28.969607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:28.969607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:28.969607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:28.969607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.969607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:28.972078 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:28.972412 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:28.972529 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:28.972659 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:28.973173 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:28.973557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.974322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:28.974322 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:28.974322 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:28.974322 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:28.974322 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:28.974322 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:28.974322 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.974322 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:28.975121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.975121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:33:28.975121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:28.975121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:28.975121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:28.975121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:28.975121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:28.975121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:28.975121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:28.975121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:28.975121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:28.975121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:33:28.975121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:28.975121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:33:28.975121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:33:28.975555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:28.975555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:33:28.975555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:28.975555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:33:28.975555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:33:28.975555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:28.975555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:33:28.975555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.975555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:33:28.976084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.976084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:28.976084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:28.976084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:28.976084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.976084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:33:28.976084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:33:28.976084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:28.976084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:28.976084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:28.976084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:28.976448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:28.976448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:28.976448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:28.976448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:28.976448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:28.976448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:28.976448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:28.976448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:28.976448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:28.976448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:28.976448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:28.976448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:28.976448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:28.976448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:28.976843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:28.976843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:28.976843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.976843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:28.977258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.977258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:28.977258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:28.977258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:28.977258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:28.977258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:28.977258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.977258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:28.977258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:28.977258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.977258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:28.977258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:28.977258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.977258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:28.977258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:28.977258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.977258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:33:28.977793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:33:28.977793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:28.977793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:28.977793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:28.977793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:28.977793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:28.977793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:28.977793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:28.977793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:28.977793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:28.977793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:28.977793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:28.977793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:28.977793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:28.977793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:28.977793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:28.977793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:28.977793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:28.977793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:28.978312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:28.978312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:28.978312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.978312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:28.980664 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:28.981021 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:28.981138 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:28.981296 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:28.981714 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:28.981933 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:28.982124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.982571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:28.982571 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:28.982571 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:28.982571 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:28.982571 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:28.982571 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:28.982571 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.982571 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:28.983452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.983452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:28.983452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:28.983452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:28.983452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.983452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:33:28.983452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:28.983452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:28.983452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:28.983452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:28.983452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:28.983452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:28.983878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:28.983878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:28.983878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:28.983878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:28.983878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:28.983878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:28.983878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:28.983878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:28.983878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:28.983878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:28.983878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:28.983878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:28.983878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:28.984232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.984232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:28.984541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.984541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:28.984541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:28.984541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:28.984541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:28.984541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:28.984541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.984541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:28.984541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:28.984541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.984541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:28.984541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:28.984541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.984541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:28.984541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:28.984541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:28.985043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:33:28.985043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:28.985043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:28.985043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:28.985043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:28.985043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:28.985043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:28.985043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:28.985043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:28.985043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:28.985043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:28.985043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:28.985043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:28.985043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:28.985043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:28.985043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:28.985043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:28.985043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:28.985043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:28.985584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:28.985584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:28.985584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:33.678404 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:33.679032 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:33.679032 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:33.679712 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:33.710011 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:33.710206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.710803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:33.710803 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:33.710803 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:33.710803 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:33.710803 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:33.710803 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:33.710803 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.710803 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:33.711591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.711591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:33.711591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:33.711591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:33.711591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.711591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:33:33.711591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:33.711591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:33.711591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:33.711591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:33.711591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:33.711591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:33.712052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:33.712052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:33.712052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:33.712052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:33.712052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:33.712052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:33.712052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:33.712052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:33.712052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:33.712052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:33.712052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:33.712052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:33.712052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:33.712451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.712451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:33.712451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.712451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:33.712451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:33.712451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:33.712451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:33.712451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:33.712451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.712451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:33.712451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:33.712451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.712451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:33.712451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:33.712451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.712944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:33.712944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:33.712944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.712944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:33:33.712944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:33.712944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:33.712944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:33.712944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:33.712944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:33.712944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:33.712944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:33.712944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:33.712944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:33.712944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:33.712944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:33.712944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:33.712944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:33.712944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:33.712944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:33.713549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:33.713549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:33.713549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:33.713549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:33.713549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.713549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:33.913180 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:33.913771 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:33.913991 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:33.915152 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:33.917701 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:33.917701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.917701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:33.917701 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:33.917701 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:33.917701 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:33.917701 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:33.917701 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:33.917701 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.917701 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:33:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:33.917967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:33.918231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:33.918231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:33.918231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:33.918231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:33.918231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:33.918231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:33.918231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:33.918231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:33.918231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:33.918231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:33.918231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:33.918231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:33.918231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:33.918484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.918484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:33.920328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.920328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:33.920328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:33.920328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:33.920328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:33.920328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:33.920328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.920328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:33.920328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:33.920328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.920328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:33.920328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:33.920328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.920328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:33.920328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:33.920328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.920717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:33:33.920717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:33.920717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:33.920717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:33.920717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:33.920717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:33.920717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:33.920717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:33.920717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:33.920717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:33.920717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:33.920717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:33.920717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:33.920717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:33.920717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:33.920717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:33.920717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:33.920717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:33.920717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:33.921083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:33.921083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.921083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:33.961000 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:33.961519 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:33.961591 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:33.961711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:33:33.962441 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:33.962786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.963644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:33.963644 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:33.963644 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:33.963644 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:33.963644 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:33.963644 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:33.963644 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.963644 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:33.964330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.964330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:33:33.964330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:33.964330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:33.964330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:33.964330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:33.964330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:33.964330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:33.964330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:33.964330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:33.964330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:33.964330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:33:33.964330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:33.964330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:33:33.964330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:33:33.964735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:33.964735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:33:33.964735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:33.964735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:33:33.964735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:33:33.964735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:33.964735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:33:33.964735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.964735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:33:33.965125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.965125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:33.965125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:33.965125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:33.965125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.965125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:33:33.965125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:33:33.965125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:33:33.965125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:33.965125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:33.965125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:33.965125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:33.965386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:33.965386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:33.965386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:33.965386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:33.965386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:33.965386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:33.965386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:33.965386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:33.965386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:33.965386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:33.965386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:33.965386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:33.965386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:33.965386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:33.965666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:33.965666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:33.965666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.965666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:33.965930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.965930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:33.965930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:33.965930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:33.965930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:33.965930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:33.965930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.965930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:33.965930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:33.965930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.965930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:33.965930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:33.965930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.965930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:33.965930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:33.965930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.966260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:33:33.966260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:33:33.966260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:33:33.966260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:33.966260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:33.966260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:33.966260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:33.966260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:33.966260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:33.966260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:33.966260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:33.966260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:33.966260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:33.966260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:33.966260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:33.966260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:33.966260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:33.966260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:33.966648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:33.966648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:33.966648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:33.966648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:33.966648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:33.966648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.966648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:33.972597 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:33.972860 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:33.972947 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:33.973051 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:33.973728 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:33.974182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.975208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:33.975208 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:33.975208 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:33.975208 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:33.975208 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:33.975208 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:33.975208 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.975208 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:33.975841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.975841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:33:33.975841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:33.975841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:33.975841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:33.975841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:33.975841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:33.975841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:33.975841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:33.975841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:33.975841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:33.975841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:33:33.975841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:33.975841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:33:33.975841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:33:33.976214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:33.976214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:33:33.976214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:33.976214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:33:33.976214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:33:33.976214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:33.976214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:33:33.976214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.976214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:33:33.976479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.976479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:33.976479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:33.976479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:33.976479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.976479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:33:33.976479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:33:33.976479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:33.976479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:33.976479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:33.976479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:33.976738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:33.976738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:33.976738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:33.976738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:33.976738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:33.976738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:33.976738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:33.976738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:33.976738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:33.976738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:33.976738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:33.976738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:33.976738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:33.976738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:33.977007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:33.977007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:33.977007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.977007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:33.977268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.977268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:33.977268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:33.977268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:33.977268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:33.977268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:33.977268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.977268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:33.977268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:33.977268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.977268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:33.977268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:33.977268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.977268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:33.977268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:33.977268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.977268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:33:33.977626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:33:33.977626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:33.977626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:33.977626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:33.977626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:33.977626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:33.977626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:33.977626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:33.977626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:33.977626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:33.977626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:33.977626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:33.977626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:33.977626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:33.977626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:33.977626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:33.977626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:33.977626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:33.977626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:33.977990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:33.977990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:33.977990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.977990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:33.981097 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:33.981333 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:33.981415 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:33.981532 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:33.982065 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:33.982372 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:33.982536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.983197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:33.983197 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:33.983197 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:33.983197 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:33.983197 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:33.983197 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:33.983197 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.983197 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:33.983821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.983821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:33.983821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:33.983821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:33.983821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.983821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:33:33.983821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:33.983821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:33.983821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:33.983821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:33.983821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:33.983821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:33.984092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:33.984092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:33.984092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:33.984092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:33.984092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:33.984092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:33.984092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:33.984092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:33.984092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:33.984092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:33.984092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:33.984092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:33.984092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:33.984326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.984326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:33.984515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.984515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:33.984515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:33.984515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:33.984515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:33.984515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:33.984515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.984515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:33.984515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:33.984515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.984515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:33.984515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:33.984515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.984515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:33.984515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:33.984515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:33.984854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:33:33.984854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:33.984854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:33.984854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:33.984854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:33.984854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:33.984854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:33.984854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:33.984854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:33.984854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:33.984854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:33.984854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:33.984854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:33.984854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:33.984854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:33.984854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:33.984854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:33.984854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:33.984854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:33.985204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:33.985204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:33.985204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:38.679187 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:38.679985 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:38.679985 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:38.680247 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:38.717671 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:38.718055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.718744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:38.718744 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:38.718744 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:38.718744 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:38.718744 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:38.718744 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:38.718744 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.718744 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:38.719907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.719907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:38.719907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:38.719907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:38.719907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.719907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:33:38.719907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:38.719907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:38.719907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:38.719907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:38.719907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:38.719907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:38.720405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:38.720405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:38.720405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:38.720405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:38.720405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:38.720405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:38.720405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:38.720405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:38.720405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:38.720405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:38.720405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:38.720405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:38.720405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:38.720869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.720869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:38.720991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.720991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:38.720991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:38.720991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:38.720991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:38.720991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:38.720991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.720991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:38.720991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:38.720991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.720991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:38.720991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:38.720991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.720991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:38.720991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:38.720991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.721734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:33:38.721734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:38.721734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:38.721734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:38.721734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:38.721734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:38.721734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:38.721734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:38.721734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:38.721734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:38.721734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:38.721734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:38.721734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:38.721734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:38.721734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:38.721734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:38.721734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:38.721734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:38.721734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:38.722556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:38.722556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.722556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:38.913768 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:38.914136 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:38.914632 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:38.915631 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:38.915631 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:38.915631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.916290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:38.916290 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:38.916290 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:38.916290 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:38.916290 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:38.916290 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:38.916290 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.916290 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:38.917271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.917271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:38.917271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:38.917271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:38.917271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.917271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:33:38.917271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:38.917271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:38.917271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:38.917271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:38.917271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:38.917271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:38.917856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:38.917856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:38.917856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:38.917856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:38.917856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:38.917856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:38.917856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:38.917856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:38.917856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:38.917856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:38.917856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:38.917856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:38.917856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:38.918644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.918644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:38.918644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.918644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:38.918644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:38.918644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:38.918644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:38.918644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:38.918644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.918644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:38.918644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:38.918644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.918644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:38.918644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:38.918644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.919211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:38.919211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:38.919211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.919211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:33:38.919211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:38.919211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:38.919211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:38.919211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:38.919211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:38.919211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:38.919211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:38.919211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:38.919211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:38.919211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:38.919211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:38.919211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:38.919211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:38.919211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:38.919211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:38.919768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:38.919768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:38.919768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:38.919768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:38.919768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.919768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:38.961639 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:38.962213 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:38.962285 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:38.962445 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:33:38.963051 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:38.963229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.963828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:38.963828 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:38.963828 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:38.963828 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:38.963828 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:38.963828 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:38.963828 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.963828 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:38.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:33:38.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:38.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:38.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:38.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:38.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:38.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:38.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:38.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:38.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:38.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:33:38.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:38.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:33:38.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:33:38.965524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:38.965524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:33:38.965524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:38.965524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:33:38.965524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:33:38.965524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:38.965524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:33:38.965524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.965524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:33:38.966503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.966503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:38.966503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:38.966503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:38.966503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.966503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:33:38.966503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:33:38.966503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:33:38.966503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:38.966503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:38.966503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:38.966503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:38.966957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:38.966957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:38.966957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:38.966957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:38.966957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:38.966957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:38.966957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:38.966957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:38.966957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:38.966957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:38.966957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:38.966957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:38.966957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:38.966957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:38.967315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:38.967315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:38.967315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.967315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:38.968805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.968805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:38.968805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:38.968805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:38.968805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:38.968805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:38.968805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.968805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:38.968805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:38.968805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.968805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:38.968805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:38.968805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.968805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:38.968805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:38.968805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.969275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:33:38.969275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:33:38.969275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:33:38.969275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:38.969275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:38.969275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:38.969275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:38.969275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:38.969275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:38.969275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:38.969275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:38.969275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:38.969275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:38.969275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:38.969275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:38.969275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:38.969275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:38.969275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:38.969796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:38.969796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:38.969796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:38.969796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:38.969796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:38.969796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.969796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:38.972896 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:38.973086 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:38.973196 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:38.973324 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:38.973786 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:38.973991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.974441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:38.974441 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:38.974441 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:38.974441 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:38.974441 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:38.974441 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:38.974441 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.974441 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:38.975192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.975192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:33:38.975192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:38.975192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:38.975192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:38.975192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:38.975192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:38.975192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:38.975192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:38.975192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:38.975192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:38.975192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:33:38.975192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:38.975192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:33:38.975192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:33:38.975612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:38.975612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:33:38.975612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:38.975612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:33:38.975612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:33:38.975612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:38.975612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:33:38.975612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.975612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:33:38.976174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.976174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:38.976174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:38.976174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:38.976174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.976174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:33:38.976174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:33:38.976174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:38.976174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:38.976174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:38.976174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:38.976505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:38.976505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:38.976505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:38.976505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:38.976505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:38.976505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:38.976505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:38.976505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:38.976505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:38.976505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:38.976505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:38.976505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:38.976505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:38.976505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:38.976932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:38.976932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:38.976932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.976932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:38.977568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.977568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:38.977568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:38.977568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:38.977568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:38.977568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:38.977568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.977568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:38.977568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:38.977568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.977568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:38.977568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:38.977568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.977568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:38.977568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:38.977568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.977568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:33:38.978242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:33:38.978242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:38.978242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:38.978242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:38.978242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:38.978242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:38.978242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:38.978242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:38.978242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:38.978242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:38.978242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:38.978242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:38.978242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:38.978242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:38.978242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:38.978242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:38.978242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:38.978242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:38.978242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:38.978762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:38.978762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:38.978762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.978762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:38.981368 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:38.981584 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:38.981688 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:38.981884 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:38.982355 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:38.982556 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:38.982701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.983163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:38.983163 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:38.983163 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:38.983163 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:38.983163 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:38.983163 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:38.983163 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.983163 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:38.984064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.984064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:38.984064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:38.984064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:38.984064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.984064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:33:38.984064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:38.984064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:38.984064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:38.984064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:38.984064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:38.984064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:38.984462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:38.984462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:38.984462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:38.984462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:38.984462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:38.984462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:38.984462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:38.984462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:38.984462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:38.984462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:38.984462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:38.984462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:38.984462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:38.984872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.984872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:38.985077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.985077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:38.985077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:38.985077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:38.985077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:38.985077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:38.985077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.985077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:38.985077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:38.985077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.985077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:38.985077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:38.985077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.985077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:38.985077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:38.985077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:38.985591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:33:38.985591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:38.985591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:38.985591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:38.985591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:38.985591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:38.985591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:38.985591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:38.985591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:38.985591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:38.985591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:38.985591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:38.985591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:38.985591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:38.985591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:38.985591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:38.985591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:38.985591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:38.985591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:38.986290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:38.986290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:38.986290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:43.680089 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:43.680896 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:43.680896 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:43.682092 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:43.720149 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:43.720358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.721065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:43.721065 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:43.721065 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:43.721065 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:43.721065 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:43.721065 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:43.721065 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.721065 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:43.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:43.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:43.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:43.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:33:43.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:43.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:43.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:43.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:43.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:43.722000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:43.722983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:43.722983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:43.722983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:43.722983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:43.722983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:43.722983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:43.722983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:43.722983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:43.722983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:43.722983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:43.722983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:43.722983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:43.722983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:43.723601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.723601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:43.723601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.723601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:43.723601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:43.723601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:43.723601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:43.723601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:43.723601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.723601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:43.723601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:43.723601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.723601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:43.723601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:43.723601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.724485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:43.724485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:43.724485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.724485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:33:43.724485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:43.724485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:43.724485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:43.724485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:43.724485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:43.724485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:43.724485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:43.724485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:43.724485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:43.724485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:43.724485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:43.724485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:43.724485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:43.724485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:43.724485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:43.725663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:43.725663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:43.725663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:43.725663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:43.725663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.725663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:43.915142 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:43.915554 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:43.915825 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:43.916833 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:43.917148 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:43.917583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.918491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:43.918491 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:43.918491 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:43.918491 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:43.918491 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:43.918491 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:43.918491 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.918491 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:43.919637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.919637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:43.919637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:43.919637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:43.919637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.919637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:33:43.919637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:43.919637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:43.919637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:43.919637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:43.919637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:43.919637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:43.920400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:43.920400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:43.920400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:43.920400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:43.920400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:43.920400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:43.920400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:43.920400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:43.920400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:43.920400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:43.920400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:43.920400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:43.920400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:43.921144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.921144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:43.921144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.921144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:43.921144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:43.921144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:43.921144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:43.921144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:43.921144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.921144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:43.921144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:43.921144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.921144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:43.921144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:43.921144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.921990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:43.921990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:43.921990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.921990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:33:43.921990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:43.921990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:43.921990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:43.921990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:43.921990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:43.921990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:43.921990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:43.921990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:43.921990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:43.921990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:43.921990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:43.921990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:43.921990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:43.921990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:43.921990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:43.922804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:43.922804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:43.922804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:43.922804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:43.922804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.922804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:43.962324 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:43.962826 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:43.963006 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:43.963183 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:33:43.963785 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:43.964092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.964843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:43.964843 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:43.964843 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:43.964843 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:43.964843 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:43.964843 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:43.964843 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.964843 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:43.965790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.965790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:33:43.965790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:43.965790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:43.965790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:43.965790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:43.965790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:43.965790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:43.965790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:43.965790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:43.965790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:43.965790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:33:43.965790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:43.965790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:33:43.965790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:33:43.966568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:43.966568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:33:43.966568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:43.966568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:33:43.966568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:33:43.966568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:43.966568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:33:43.966568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.966568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:33:43.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:43.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:43.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:43.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:33:43.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:33:43.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:33:43.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:43.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:43.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:43.967054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:43.967540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:43.967540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:43.967540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:43.967540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:43.967540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:43.967540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:43.967540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:43.967540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:43.967540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:43.967540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:43.967540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:43.967540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:43.967540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:43.967540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:43.968134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:43.968134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:43.968134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.968134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:43.968501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.968501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:43.968501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:43.968501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:43.968501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:43.968501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:43.968501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.968501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:43.968501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:43.968501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.968501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:43.968501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:43.968501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.968501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:43.968501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:43.968501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.969037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:33:43.969037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:33:43.969037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:33:43.969037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:43.969037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:43.969037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:43.969037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:43.969037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:43.969037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:43.969037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:43.969037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:43.969037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:43.969037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:43.969037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:43.969037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:43.969037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:43.969037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:43.969037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:43.969534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:43.969534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:43.969534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:43.969534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:43.969534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:43.969534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.969534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:43.973467 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:43.973848 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:43.973992 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:43.974168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:43.974686 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:43.975071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.975705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:43.975705 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:43.975705 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:43.975705 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:43.975705 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:43.975705 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:43.975705 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.975705 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:43.976543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.976543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:33:43.976543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:43.976543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:43.976543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:43.976543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:43.976543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:43.976543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:43.976543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:43.976543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:43.976543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:43.976543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:33:43.976543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:43.976543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:33:43.976543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:33:43.977166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:43.977166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:33:43.977166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:43.977166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:33:43.977166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:33:43.977166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:43.977166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:33:43.977166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.977166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:33:43.977770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.977770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:43.977770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:43.977770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:43.977770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.977770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:33:43.977770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:33:43.977770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:43.977770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:43.977770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:43.977770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:43.978274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:43.978274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:43.978274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:43.978274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:43.978274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:43.978274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:43.978274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:43.978274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:43.978274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:43.978274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:43.978274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:43.978274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:43.978274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:43.978274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:43.978886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:43.978886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:43.978886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.978886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:43.979381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.979381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:43.979381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:43.979381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:43.979381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:43.979381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:43.979381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.979381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:43.979381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:43.979381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.979381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:43.979381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:43.979381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.979381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:43.979381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:43.979381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.979381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:33:43.979944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:33:43.979944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:43.979944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:43.979944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:43.979944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:43.979944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:43.979944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:43.979944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:43.979944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:43.979944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:43.979944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:43.979944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:43.979944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:43.979944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:43.979944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:43.979944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:43.979944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:43.979944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:43.979944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:43.980463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:43.980463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:43.980463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.980463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:43.981859 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:43.982174 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:43.982323 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:43.982525 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:43.983049 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:43.983393 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:43.983621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.984293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:43.984293 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:43.984293 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:43.984293 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:43.984293 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:43.984293 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:43.984293 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.984293 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:43.985222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.985222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:43.985222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:43.985222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:43.985222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.985222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:33:43.985222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:43.985222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:43.985222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:43.985222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:43.985222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:43.985222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:43.985778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:43.985778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:43.985778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:43.985778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:43.985778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:43.985778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:43.985778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:43.985778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:43.985778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:43.985778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:43.985778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:43.985778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:43.985778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:43.986344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.986344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:43.986596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.986596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:43.986596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:43.986596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:43.986596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:43.986596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:43.986596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.986596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:43.986596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:43.986596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.986596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:43.986596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:43.986596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.986596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:43.986596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:43.986596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:43.987193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:33:43.987193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:43.987193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:43.987193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:43.987193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:43.987193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:43.987193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:43.987193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:43.987193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:43.987193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:43.987193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:43.987193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:43.987193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:43.987193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:43.987193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:43.987193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:43.987193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:43.987193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:43.987193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:43.987733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:43.987733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:43.987733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:48.682636 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:48.683400 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:48.683473 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:48.684676 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:48.712484 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:48.713013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.713867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:48.713867 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:48.713867 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:48.713867 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:48.713867 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:48.713867 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:48.713867 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.713867 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:48.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:48.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:48.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:48.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:33:48.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:48.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:48.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:48.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:48.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:48.714583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:48.715289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:48.715289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:48.715289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:48.715289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:48.715289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:48.715289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:48.715289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:48.715289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:48.715289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:48.715289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:48.715289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:48.715289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:48.715289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:48.716027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.716027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:48.716027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.716027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:48.716027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:48.716027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:48.716027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:48.716027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:48.716027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.716027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:48.716027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:48.716027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.716027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:48.716027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:48.716027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.717065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:48.717065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:48.717065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.717065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:33:48.717065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:48.717065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:48.717065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:48.717065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:48.717065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:48.717065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:48.717065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:48.717065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:48.717065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:48.717065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:48.717065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:48.717065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:48.717065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:48.717065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:48.717065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:48.718082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:48.718082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:48.718082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:48.718082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:48.718082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.718082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:48.915619 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:48.916457 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:48.916556 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:48.917707 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:48.918119 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:48.918373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.919578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:48.919578 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:48.919578 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:48.919578 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:48.919578 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:48.919578 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:48.919578 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.919578 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:48.920535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.920535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:48.920535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:48.920535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:48.920535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.920535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:33:48.920535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:48.920535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:48.920535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:48.920535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:48.920535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:48.920535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:48.921202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:48.921202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:48.921202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:48.921202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:48.921202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:48.921202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:48.921202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:48.921202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:48.921202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:48.921202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:48.921202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:48.921202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:48.921202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:48.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:48.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:48.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:48.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:48.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:48.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:48.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:48.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:48.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:48.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:48.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:48.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:48.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:33:48.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:48.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:48.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:48.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:48.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:48.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:48.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:48.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:48.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:48.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:48.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:48.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:48.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:48.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:48.922591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:48.923545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:48.923545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:48.923545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:48.923545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:48.923545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.923545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:48.963074 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:48.963666 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:48.964061 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:48.964125 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:33:48.964943 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:48.965285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.966339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:48.966339 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:48.966339 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:48.966339 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:48.966339 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:48.966339 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:48.966339 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.966339 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:48.967563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.967563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:33:48.967563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:48.967563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:48.967563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:48.967563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:48.967563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:48.967563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:48.967563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:48.967563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:48.967563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:48.967563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:33:48.967563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:48.967563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:33:48.967563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:33:48.967974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:48.967974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:33:48.967974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:48.967974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:33:48.967974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:33:48.967974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:48.967974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:33:48.967974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.967974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:33:48.969232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.969232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:48.969232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:48.969232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:48.969232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.969232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:33:48.969232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:33:48.969232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:33:48.969232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:48.969232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:48.969232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:48.969232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:48.969674 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:48.969674 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:48.969674 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:48.969674 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:48.969674 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:48.969674 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:48.969674 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:48.969674 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:48.969674 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:48.969674 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:48.969674 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:48.969674 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:48.969674 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:48.969674 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:48.970073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:48.970073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:48.970073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.970073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:48.970212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.970212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:48.970212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:48.970212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:48.970212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:48.970212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:48.970212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.970212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:48.970212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:48.970212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.970212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:48.970212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:48.970212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.970212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:48.970212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:48.970212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.970952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:33:48.970952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:33:48.970952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:33:48.970952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:48.970952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:48.970952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:48.970952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:48.970952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:48.970952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:48.970952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:48.970952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:48.970952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:48.970952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:48.970952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:48.970952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:48.970952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:48.970952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:48.970952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:48.971473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:48.971473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:48.971473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:48.971473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:48.971473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:48.971473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.971473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:48.973928 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:48.974439 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:48.974625 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:48.974878 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:48.975294 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:48.975698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.976361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:48.976361 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:48.976361 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:48.976361 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:48.976361 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:48.976361 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:48.976361 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.976361 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:48.977690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.977690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:33:48.977690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:48.977690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:48.977690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:48.977690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:48.977690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:48.977690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:48.977690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:48.977690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:48.977690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:48.977690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:33:48.977690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:48.977690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:33:48.977690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:33:48.978120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:48.978120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:33:48.978120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:48.978120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:33:48.978120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:33:48.978120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:48.978120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:33:48.978120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.978120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:33:48.979334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.979334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:48.979334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:48.979334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:48.979334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.979334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:33:48.979334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:33:48.979334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:48.979334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:48.979334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:48.979334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:48.979668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:48.979668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:48.979668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:48.979668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:48.979668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:48.979668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:48.979668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:48.979668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:48.979668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:48.979668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:48.979668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:48.979668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:48.979668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:48.979668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:48.980100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:48.980100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:48.980100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.980100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:48.981067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.981067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:48.981067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:48.981067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:48.981067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:48.981067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:48.981067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.981067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:48.981067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:48.981067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.981067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:48.981067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:48.981067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.981067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:48.981067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:48.981067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.981067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:33:48.981565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:33:48.981565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:48.981565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:48.981565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:48.981565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:48.981565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:48.981565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:48.981565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:48.981565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:48.981565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:48.981565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:48.981565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:48.981565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:48.981565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:48.981565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:48.981565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:48.981565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:48.981565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:48.981565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:48.982108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:48.982108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:48.982108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.982108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:48.982228 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:48.982470 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:48.982607 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:48.982878 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:48.983181 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:48.983418 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:48.983577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.984103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:48.984103 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:48.984103 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:48.984103 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:48.984103 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:48.984103 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:48.984103 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.984103 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:48.985003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.985003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:48.985003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:48.985003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:48.985003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.985003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:33:48.985003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:48.985003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:48.985003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:48.985003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:48.985003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:48.985003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:48.985358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:48.985358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:48.985358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:48.985358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:48.985358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:48.985358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:48.985358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:48.985358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:48.985358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:48.985358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:48.985358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:48.985358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:48.985358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:48.985729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.985729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:48.986016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.986016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:48.986016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:48.986016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:48.986016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:48.986016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:48.986016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.986016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:48.986016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:48.986016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.986016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:48.986016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:48.986016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.986016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:48.986016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:48.986016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:48.986470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:33:48.986470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:48.986470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:48.986470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:48.986470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:48.986470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:48.986470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:48.986470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:48.986470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:48.986470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:48.986470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:48.986470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:48.986470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:48.986470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:48.986470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:48.986470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:48.986470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:48.986470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:48.986470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:48.987008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:48.987008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:48.987008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:53.683529 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:53.684217 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:53.684399 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:53.686008 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:53.722142 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:53.723315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.724182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:53.724182 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:53.724182 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:53.724182 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:53.724182 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:53.724182 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:53.724182 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.724182 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:53.725129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.725129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:53.725129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:53.725129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:53.725129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.725129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:33:53.725129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:53.725129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:53.725129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:53.725129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:53.725129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:53.725129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:53.725421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:53.725421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:53.725421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:53.725421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:53.725421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:53.725421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:53.725421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:53.725421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:53.725421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:53.725421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:53.725421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:53.725421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:53.725421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:53.725667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.725667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:53.726226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.726226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:53.726226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:53.726226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:53.726226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:53.726226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:53.726226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.726226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:53.726226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:53.726226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.726226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:53.726226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:53.726226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.726226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:53.726226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:53.726226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.726564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:33:53.726564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:53.726564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:53.726564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:53.726564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:53.726564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:53.726564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:53.726564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:53.726564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:53.726564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:53.726564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:53.726564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:53.726564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:53.726564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:53.726564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:53.726564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:53.726564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:53.726564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:53.726564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:53.726957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:53.726957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.726957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:53.916443 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:53.917181 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:53.917457 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:53.918639 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:53.919108 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:53.919420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.921159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:53.921159 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:53.921159 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:53.921159 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:53.921159 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:53.921159 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:53.921159 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.921159 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:53.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:53.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:53.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:53.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:33:53.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:53.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:53.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:53.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:53.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:53.921852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:53.922399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:53.922399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:53.922399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:53.922399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:53.922399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:53.922399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:53.922399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:53.922399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:53.922399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:53.922399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:53.922399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:53.922399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:53.922399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:53.922985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.922985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:53.923538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.923538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:53.923538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:53.923538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:53.923538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:53.923538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:53.923538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.923538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:53.923538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:53.923538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.923538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:53.923538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:53.923538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.923538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:53.923538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:53.923538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.924349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:33:53.924349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:53.924349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:53.924349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:53.924349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:53.924349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:53.924349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:53.924349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:53.924349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:53.924349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:53.924349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:53.924349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:53.924349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:53.924349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:53.924349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:53.924349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:53.924349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:53.924349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:53.924349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:53.925230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:53.925230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.925230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:53.963757 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:53.964296 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:53.964528 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:53.964887 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:33:53.966801 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:53.967242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.967997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:53.967997 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:53.967997 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:53.967997 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:53.967997 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:53.967997 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:53.967997 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.967997 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:53.968873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.968873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:33:53.968873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:53.968873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:53.968873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:53.968873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:53.968873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:53.968873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:53.968873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:53.968873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:53.968873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:53.968873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:33:53.968873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:53.968873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:33:53.968873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:33:53.969513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:53.969513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:33:53.969513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:53.969513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:33:53.969513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:33:53.969513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:53.969513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:33:53.969513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.969513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:33:53.969817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.969817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:53.969817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:53.969817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:53.969817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.969817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:33:53.969817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:33:53.969817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:33:53.969817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:53.969817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:53.969817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:53.969817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:53.970125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:53.970125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:53.970125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:53.970125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:53.970125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:53.970125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:53.970125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:53.970125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:53.970125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:53.970125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:53.970125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:53.970125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:53.970125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:53.970125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:53.970486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:53.970486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:53.970486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.970486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:53.970936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.970936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:53.970936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:53.970936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:53.970936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:53.970936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:53.970936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.970936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:53.970936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:53.970936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.970936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:53.970936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:53.970936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.970936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:53.970936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:53.970936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.971339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:33:53.971339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:33:53.971339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:33:53.971339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:53.971339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:53.971339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:53.971339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:53.971339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:53.971339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:53.971339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:53.971339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:53.971339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:53.971339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:53.971339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:53.971339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:53.971339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:53.971339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:53.971339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:53.971833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:53.971833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:53.971833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:53.971833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:53.971833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:53.971833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.971833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:53.974398 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:53.974757 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:53.974883 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:53.975017 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:53.975296 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:53.975537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.977845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:53.977845 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:53.977845 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:53.977845 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:53.977845 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:53.977845 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:53.977845 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.977845 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:53.977845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.977845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:33:53.977845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:53.977845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:53.977845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:53.977845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:53.977845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:53.977845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:53.977845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:53.977845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:53.978837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:53.978837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:33:53.978837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:53.978837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:33:53.978837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:33:53.978837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:53.978837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:33:53.978837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:53.978837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:33:53.978837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:33:53.978837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:53.978837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:33:53.978837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.978837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:33:53.978837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.979613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:53.979613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:53.979613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:53.979613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.979613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:33:53.979613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:33:53.979613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:53.979613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:53.979613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:53.979613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:53.979613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:53.979613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:53.979613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:53.980235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:53.980235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:53.980235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:53.980235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:53.980235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:53.980235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:53.980235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:53.980235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:53.980235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:53.980235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:53.980235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:53.980235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:53.980235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:53.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:53.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:53.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:53.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:53.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:53.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:53.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:53.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:53.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:53.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:53.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.981579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:53.981579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:53.981579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.981579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:33:53.981579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:33:53.981579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:53.981579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:53.981579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:53.981579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:53.981579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:53.981579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:53.981579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:53.981579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:53.981579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:53.981579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:53.981579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:53.981579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:53.981579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:53.981579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:53.982186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:53.982186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:53.982186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:53.982186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:53.982186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:53.982186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:53.982186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.982186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:53.982545 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:53.982806 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:53.982930 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:53.983097 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:53.983577 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:53.983977 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:53.984213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.985021 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:53.985021 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:53.985021 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:53.985021 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:53.985021 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:53.985021 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:53.985021 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.985021 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:53.985900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.985900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:53.985900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:53.985900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:53.985900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.985900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:33:53.985900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:53.985900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:53.985900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:53.985900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:53.985900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:53.985900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:53.986206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:53.986206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:53.986206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:53.986206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:53.986206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:53.986206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:53.986206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:53.986206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:53.986206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:53.986206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:53.986206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:53.986206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:53.986206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:53.986546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.986546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:53.986910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.986910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:53.986910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:53.986910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:53.986910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:53.986910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:53.986910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.986910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:53.986910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:53.986910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.986910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:53.986910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:53.986910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.986910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:53.986910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:53.986910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:53.987366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:33:53.987366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:53.987366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:53.987366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:53.987366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:53.987366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:53.987366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:53.987366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:53.987366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:53.987366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:53.987366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:53.987366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:53.987366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:53.987366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:53.987366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:53.987366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:53.987366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:53.987366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:53.987366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:53.987936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:53.987936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:53.987936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:58.684532 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:58.685623 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:58.685623 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:58.686792 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:58.724657 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:58.725123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.726007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:58.726007 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:58.726007 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:58.726007 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:58.726007 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:58.726007 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:58.726007 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.726007 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:58.727171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.727171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:58.727171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:58.727171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:58.727171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.727171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:33:58.727171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:58.727171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:58.727171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:58.727171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:58.727171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:58.727171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:58.727770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:58.727770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:58.727770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:58.727770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:58.727770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:58.727770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:58.727770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:58.727770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:58.727770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:58.727770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:58.727770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:58.727770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:58.727770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:58.728439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.728439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:58.728439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.728439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:58.728439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:58.728439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:58.728439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:58.728439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:58.728439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.728439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:58.728439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:58.728439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.728439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:58.728439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:58.728439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.729173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:58.729173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:58.729173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.729173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:33:58.729173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:58.729173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:58.729173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:58.729173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:58.729173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:58.729173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:58.729173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:58.729173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:58.729173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:58.729173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:58.729173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:58.729173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:58.729173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:58.729173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:58.729173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:58.729866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:58.729866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:58.729866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:58.729866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:58.729866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.729866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:58.917134 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:58.917997 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:58.918199 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:58.919231 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:58.919547 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:58.919877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.920854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:58.920854 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:58.920854 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:58.920854 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:58.920854 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:58.920854 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:58.920854 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.920854 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:58.922430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.922430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:58.922430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:58.922430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:58.922430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.922430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:33:58.922430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:58.922430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:58.922430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:58.922430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:58.922430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:58.922430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:58.923116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:58.923116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:58.923116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:58.923116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:58.923116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:58.923116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:58.923116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:58.923116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:58.923116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:58.923116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:58.923116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:58.923116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:58.923116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:58.923784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.923784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:58.924547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.924547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:58.924547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:58.924547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:58.924547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:58.924547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:58.924547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.924547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:58.924547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:58.924547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.924547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:58.924547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:58.924547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.924547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:58.924547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:58.924547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.925556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:33:58.925556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:58.925556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:58.925556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:58.925556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:58.925556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:58.925556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:58.925556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:58.925556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:58.925556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:58.925556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:58.925556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:58.925556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:58.925556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:58.925556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:58.925556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:58.925556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:58.925556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:58.925556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:58.926512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:58.926512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.926512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:58.964843 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:58.965335 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:58.965473 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:58.965677 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:33:58.966504 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:58.966893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.967781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:58.967781 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:58.967781 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:58.967781 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:58.967781 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:58.967781 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:58.967781 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.967781 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:58.969060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.969060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:33:58.969060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:58.969060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:58.969060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:58.969060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:58.969060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:58.969060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:58.969060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:58.969060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:58.969060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:58.969060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:33:58.969060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:58.969060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:33:58.969060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:33:58.969877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:58.969877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:33:58.969877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:58.969877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:33:58.969877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:33:58.969877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:58.969877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:33:58.969877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.969877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:33:58.970753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.970753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:58.970753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:58.970753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:58.970753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.970753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:33:58.970753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:33:58.970753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:33:58.970753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:58.970753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:58.970753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:58.970753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:58.971313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:58.971313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:58.971313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:58.971313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:58.971313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:58.971313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:58.971313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:58.971313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:58.971313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:58.971313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:58.971313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:58.971313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:58.971313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:58.971313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:58.971911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:58.971911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:58.971911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.971911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:58.972858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.972858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:58.972858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:58.972858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:58.972858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:58.972858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:58.972858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.972858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:58.972858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:58.972858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.972858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:58.972858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:58.972858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.972858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:58.972858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:58.972858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.973548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:33:58.973548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:33:58.973548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:33:58.973548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:58.973548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:58.973548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:58.973548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:58.973548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:58.973548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:58.973548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:58.973548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:58.973548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:58.973548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:58.973548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:58.973548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:58.973548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:58.973548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:58.973548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:58.974381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:58.974381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:58.974381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:58.974381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:58.974381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:58.974381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.974381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:58.975002 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:58.975424 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:58.975590 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:58.975855 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:58.976434 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:58.976814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.977509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:58.977509 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:58.977509 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:58.977509 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:58.977509 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:58.977509 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:58.977509 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.977509 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:58.978391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.978391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:33:58.978391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:58.978391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:58.978391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:58.978391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:58.978391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:58.978391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:58.978391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:58.978391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:33:58.978391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:58.978391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:33:58.978391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:58.978391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:33:58.978391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:33:58.978832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:58.978832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:33:58.978832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:58.978832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:33:58.978832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:33:58.978832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:58.978832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:33:58.978832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.978832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:33:58.979490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.979490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:58.979490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:58.979490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:58.979490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.979490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:33:58.979490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:33:58.979490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:58.979490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:58.979490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:58.979490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:58.979859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:58.979859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:58.979859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:58.979859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:58.979859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:58.979859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:58.979859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:58.979859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:58.979859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:58.979859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:58.979859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:58.979859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:58.979859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:58.979859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:58.980241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:58.980241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:58.980241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.980241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:58.980884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.980884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:58.980884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:58.980884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:58.980884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:58.980884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:58.980884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.980884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:58.980884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:58.980884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.980884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:58.980884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:58.980884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.980884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:58.980884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:58.980884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.980884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:33:58.981392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:33:58.981392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:33:58.981392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:33:58.981392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:58.981392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:58.981392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:58.981392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:58.981392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:58.981392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:58.981392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:58.981392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:58.981392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:58.981392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:58.981392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:58.981392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:58.981392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:58.981392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:58.981392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:58.981392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:58.982050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:58.982050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:58.982050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.982050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:33:58.982982 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:33:58.983338 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:33:58.983608 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:33:58.983837 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:33:58.984298 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:58.984641 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:33:58.984900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.985565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:33:58.985565 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:58.985565 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:33:58.985565 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:33:58.985565 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:58.985565 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:33:58.985565 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.985565 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:33:58.986567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.986567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:33:58.986567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:58.986567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:33:58.986567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.986567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:33:58.986567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:58.986567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:58.986567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:58.986567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:58.986567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:58.986567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:58.986982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:33:58.986982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:58.986982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:33:58.986982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:58.986982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:33:58.986982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:33:58.986982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:58.986982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:33:58.986982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:58.986982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:33:58.986982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:33:58.986982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:58.986982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:33:58.987348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.987348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:33:58.987951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.987951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:33:58.987951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:33:58.987951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:33:58.987951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:33:58.987951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:33:58.987951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.987951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:33:58.987951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:33:58.987951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.987951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:33:58.987951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:33:58.987951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.987951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:33:58.987951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:33:58.987951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:33:58.988442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:33:58.988442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:33:58.988442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:58.988442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:58.988442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:33:58.988442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:33:58.988442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:33:58.988442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:33:58.988442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:33:58.988442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:33:58.988442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:33:58.988442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:33:58.988442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:33:58.988442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:33:58.988442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:33:58.988442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:33:58.988442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:33:58.988442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:33:58.988442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:33:58.989020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:33:58.989020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:33:58.989020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:03.685309 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:03.685996 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:03.686133 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:03.687001 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:03.722609 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:03.722838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.723571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:03.723571 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:03.723571 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:03.723571 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:03.723571 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:03.723571 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:03.723571 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.723571 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:03.724744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.724744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:03.724744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:03.724744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:03.724744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.724744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:34:03.724744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:03.724744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:03.724744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:03.724744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:03.724744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:03.724744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:03.725184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:03.725184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:03.725184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:03.725184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:03.725184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:03.725184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:03.725184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:03.725184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:03.725184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:03.725184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:03.725184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:03.725184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:03.725184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:03.725573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.725573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:03.725573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.725573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:03.725573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:03.725573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:03.725573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:03.725573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:03.725573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.725573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:03.725573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:03.725573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.725573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:03.725573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:03.725573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.726024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:03.726024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:03.726024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.726024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:34:03.726024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:03.726024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:03.726024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:03.726024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:03.726024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:03.726024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:03.726024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:03.726024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:03.726024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:03.726024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:03.726024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:03.726024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:03.726024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:03.726024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:03.726024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:03.726544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:03.726544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:03.726544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:03.726544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:03.726544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.726544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:03.918357 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:03.918947 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:03.919123 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:03.920178 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:03.920520 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:03.920801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.921606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:03.921606 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:03.921606 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:03.921606 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:03.921606 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:03.921606 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:03.921606 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.921606 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:03.923162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.923162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:03.923162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:03.923162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:03.923162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.923162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:34:03.923162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:03.923162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:03.923162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:03.923162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:03.923162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:03.923162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:03.924382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:03.924382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:03.924382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:03.924382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:03.924382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:03.924382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:03.924382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:03.924382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:03.924382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:03.924382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:03.924382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:03.924382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:03.924382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:03.925015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.925015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:03.925015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.925015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:03.925015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:03.925015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:03.925015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:03.925015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:03.925015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.925015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:03.925015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:03.925015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.925015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:03.925015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:03.925015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.926009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:03.926009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:03.926009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.926009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:34:03.926009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:03.926009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:03.926009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:03.926009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:03.926009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:03.926009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:03.926009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:03.926009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:03.926009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:03.926009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:03.926009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:03.926009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:03.926009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:03.926009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:03.926009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:03.926850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:03.926850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:03.926850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:03.926850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:03.926850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.926850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:03.965885 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:03.966599 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:03.966724 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:03.966939 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:34:03.967745 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:03.968077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.968945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:03.968945 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:03.968945 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:03.968945 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:03.968945 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:03.968945 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:03.968945 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.968945 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:03.969798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.969798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:34:03.969798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:03.969798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:03.969798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:03.969798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:03.969798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:03.969798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:03.969798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:03.969798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:03.969798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:03.969798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:34:03.969798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:03.969798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:34:03.969798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:34:03.970523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:03.970523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:34:03.970523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:03.970523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:34:03.970523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:34:03.970523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:03.970523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:34:03.970523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.970523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:34:03.970978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.970978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:03.970978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:03.970978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:03.970978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.970978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:34:03.970978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:34:03.970978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:34:03.970978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:03.970978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:03.970978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:03.970978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:03.971537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:03.971537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:03.971537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:03.971537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:03.971537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:03.971537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:03.971537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:03.971537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:03.971537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:03.971537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:03.971537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:03.971537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:03.971537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:03.971537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:03.972253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:03.972253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:03.972253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.972253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:03.972253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.972253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:03.972253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:03.972253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:03.972253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:03.972253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:03.972253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.972253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:03.972253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:03.972253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.972869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:03.972869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:03.972869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.972869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:03.972869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:03.972869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.972869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:34:03.972869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:34:03.972869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:34:03.972869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:03.972869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:03.972869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:03.972869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:03.972869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:03.972869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:03.972869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:03.972869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:03.972869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:03.972869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:03.973371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:03.973371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:03.973371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:03.973371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:03.973371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:03.973371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:03.973371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:03.973371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:03.973371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:03.973371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:03.973371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.973371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:03.975431 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:03.975817 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:03.975931 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:03.976058 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:03.976447 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:03.976765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.977306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:03.977306 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:03.977306 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:03.977306 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:03.977306 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:03.977306 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:03.977306 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.977306 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:03.978047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.978047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:34:03.978047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:03.978047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:03.978047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:03.978047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:03.978047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:03.978047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:03.978047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:03.978047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:03.978047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:03.978047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:34:03.978047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:03.978047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:34:03.978047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:34:03.978433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:03.978433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:34:03.978433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:03.978433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:34:03.978433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:34:03.978433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:03.978433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:34:03.978433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.978433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:34:03.978908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.978908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:03.978908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:03.978908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:03.978908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.978908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:34:03.978908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:34:03.978908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:03.978908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:03.978908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:03.978908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:03.979216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:03.979216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:03.979216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:03.979216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:03.979216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:03.979216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:03.979216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:03.979216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:03.979216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:03.979216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:03.979216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:03.979216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:03.979216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:03.979216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:03.979632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:03.979632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:03.979632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.979632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:03.980054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.980054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:03.980054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:03.980054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:03.980054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:03.980054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:03.980054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.980054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:03.980054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:03.980054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.980054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:03.980054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:03.980054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.980054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:03.980054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:03.980054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.980054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:34:03.980495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:34:03.980495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:03.980495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:03.980495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:03.980495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:03.980495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:03.980495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:03.980495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:03.980495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:03.980495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:03.980495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:03.980495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:03.980495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:03.980495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:03.980495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:03.980495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:03.980495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:03.980495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:03.980495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:03.981025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:03.981025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:03.981025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.981025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:03.983439 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:03.983807 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:03.983957 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:03.984120 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:03.984486 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:03.984806 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:03.985016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.985635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:03.985635 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:03.985635 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:03.985635 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:03.985635 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:03.985635 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:03.985635 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.985635 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:03.986543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.986543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:03.986543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:03.986543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:03.986543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.986543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:34:03.986543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:03.986543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:03.986543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:03.986543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:03.986543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:03.986543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:03.986906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:03.986906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:03.986906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:03.986906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:03.986906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:03.986906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:03.986906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:03.986906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:03.986906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:03.986906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:03.986906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:03.986906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:03.986906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:03.987256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.987256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:03.987801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.987801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:03.987801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:03.987801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:03.987801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:03.987801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:03.987801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.987801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:03.987801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:03.987801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.987801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:03.987801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:03.987801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.987801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:03.987801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:03.987801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:03.988218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:34:03.988218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:03.988218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:03.988218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:03.988218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:03.988218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:03.988218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:03.988218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:03.988218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:03.988218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:03.988218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:03.988218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:03.988218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:03.988218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:03.988218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:03.988218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:03.988218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:03.988218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:03.988218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:03.988774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:03.988774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:03.988774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:08.686124 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:08.687061 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:08.687061 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:08.687926 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:08.722372 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=240 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:34:08.725908 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:08.726305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.727174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:08.727174 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:08.727174 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:08.727174 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:08.727174 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:08.727174 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:08.727174 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.727174 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:08.728520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.728520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:08.728520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:08.728520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:08.728520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.728520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:34:08.728520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:08.728520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:08.728520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:08.728520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:08.728520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:08.728520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:08.728916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:08.728916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:08.728916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:08.728916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:08.728916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:08.728916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:08.728916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:08.728916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:08.728916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:08.728916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:08.728916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:08.728916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:08.728916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:08.729302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.729302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:08.729774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.729774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:08.729774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:08.729774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:08.729774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:08.729774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:08.729774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.729774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:08.729774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:08.729774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.729774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:08.729774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:08.729774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.729774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:08.729774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:08.729774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.730224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:34:08.730224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:08.730224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:08.730224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:08.730224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:08.730224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:08.730224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:08.730224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:08.730224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:08.730224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:08.730224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:08.730224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:08.730224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:08.730224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:08.730224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:08.730224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:08.730224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:08.730224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:08.730224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:08.730789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:08.730789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.730789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:08.919060 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:08.919564 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:08.919718 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:08.920985 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:08.920985 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:08.921056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.921661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:08.921661 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:08.921661 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:08.921661 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:08.921661 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:08.921661 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:08.921661 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.921661 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:08.922752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.922752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:08.922752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:08.922752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:08.922752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.922752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:34:08.922752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:08.922752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:08.922752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:08.922752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:08.922752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:08.922752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:08.923332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:08.923332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:08.923332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:08.923332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:08.923332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:08.923332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:08.923332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:08.923332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:08.923332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:08.923332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:08.923332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:08.923332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:08.923332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:08.923896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.923896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:08.923896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.923896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:08.923896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:08.923896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:08.923896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:08.923896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:08.923896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.923896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:08.923896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:08.923896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.923896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:08.923896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:08.923896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.924585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:08.924585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:08.924585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.924585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:34:08.924585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:08.924585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:08.924585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:08.924585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:08.924585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:08.924585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:08.924585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:08.924585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:08.924585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:08.924585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:08.924585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:08.924585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:08.924585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:08.924585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:08.924585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:08.925226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:08.925226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:08.925226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:08.925226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:08.925226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.925226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:08.966455 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:08.967083 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:08.967268 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:08.967448 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:34:08.968058 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:08.968443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.969185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:08.969185 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:08.969185 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:08.969185 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:08.969185 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:08.969185 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:08.969185 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.969185 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:08.970431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.970431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:34:08.970431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:08.970431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:08.970431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:08.970431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:08.970431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:08.970431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:08.970431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:08.970431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:08.970431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:08.970431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:34:08.970431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:08.970431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:34:08.970431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:34:08.971129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:08.971129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:34:08.971129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:08.971129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:34:08.971129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:34:08.971129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:08.971129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:34:08.971129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.971129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:34:08.971693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.971693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:08.971693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:08.971693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:08.971693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.971693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:34:08.971693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:34:08.971693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:34:08.971693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:08.971693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:08.971693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:08.971693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:08.972327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:08.972327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:08.972327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:08.972327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:08.972327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:08.972327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:08.972327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:08.972327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:08.972327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:08.972327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:08.972327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:08.972327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:08.972327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:08.972327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:08.972914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:08.972914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:08.972914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.972914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:08.973584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.973584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:08.973584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:08.973584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:08.973584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:08.973584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:08.973584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.973584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:08.973584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:08.973584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.973584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:08.973584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:08.973584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.973584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:08.973584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:08.973584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.974043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:34:08.974043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:34:08.974043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:34:08.974043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:08.974043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:08.974043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:08.974043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:08.974043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:08.974043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:08.974043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:08.974043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:08.974043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:08.974043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:08.974043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:08.974043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:08.974043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:08.974043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:08.974043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:08.974566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:08.974566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:08.974566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:08.974566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:08.974566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:08.974566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.974566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:08.975799 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:08.976088 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:08.976199 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:08.976326 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:08.976827 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:08.977186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.977923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:08.977923 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:08.977923 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:08.977923 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:08.977923 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:08.977923 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:08.977923 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.977923 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:08.978685 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.978685 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:34:08.978685 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:08.978685 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:08.978685 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:08.978685 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:08.978685 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:08.978685 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:08.978685 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:08.978685 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:08.978685 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:08.978685 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:34:08.978685 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:08.978685 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:34:08.978685 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:34:08.979101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:08.979101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:34:08.979101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:08.979101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:34:08.979101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:34:08.979101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:08.979101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:34:08.979101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.979101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:34:08.979587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.979587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:08.979587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:08.979587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:08.979587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.979587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:34:08.979587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:34:08.979587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:08.979587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:08.979587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:08.979587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:08.979990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:08.979990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:08.979990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:08.979990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:08.979990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:08.979990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:08.979990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:08.979990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:08.979990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:08.979990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:08.979990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:08.979990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:08.979990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:08.979990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:08.980459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:08.980459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:08.980459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.980459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:08.980741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.980741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:08.980741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:08.980741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:08.980741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:08.980741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:08.980741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.980741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:08.980741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:08.980741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.980741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:08.980741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:08.980741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.980741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:08.980741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:08.980741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.980741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:34:08.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:34:08.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:08.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:08.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:08.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:08.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:08.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:08.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:08.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:08.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:08.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:08.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:08.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:08.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:08.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:08.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:08.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:08.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:08.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:08.981772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:08.981772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:08.981772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.981772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:08.983898 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:08.984195 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:08.984304 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:08.984457 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:08.984908 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:08.985451 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:08.985697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.986933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:08.986933 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:08.986933 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:08.986933 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:08.986933 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:08.986933 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:08.986933 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.986933 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:08.987798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.987798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:08.987798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:08.987798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:08.987798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.987798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:34:08.987798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:08.987798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:08.987798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:08.987798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:08.987798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:08.987798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:08.988228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:08.988228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:08.988228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:08.988228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:08.988228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:08.988228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:08.988228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:08.988228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:08.988228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:08.988228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:08.988228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:08.988228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:08.988228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:08.988603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.988603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:08.988822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.988822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:08.988822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:08.988822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:08.988822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:08.988822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:08.988822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.988822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:08.988822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:08.988822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.988822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:08.988822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:08.988822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.988822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:08.988822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:08.988822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:08.989300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:34:08.989300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:08.989300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:08.989300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:08.989300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:08.989300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:08.989300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:08.989300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:08.989300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:08.989300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:08.989300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:08.989300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:08.989300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:08.989300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:08.989300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:08.989300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:08.989300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:08.989300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:08.989300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:08.989928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:08.989928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:08.989928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:13.686765 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:13.687424 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:13.687568 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:13.688353 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:13.723421 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:13.723421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.723606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:13.723606 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:13.723606 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:13.723606 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:13.723606 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:13.723606 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:13.723606 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.723606 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:13.724155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.724155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:13.724155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:13.724155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:13.724155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.724155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:34:13.724155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:13.724155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:13.724155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:13.724155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:13.724155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:13.724155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:13.724582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:13.724582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:13.724582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:13.724582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:13.724582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:13.724582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:13.724582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:13.724582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:13.724582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:13.724582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:13.724582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:13.724582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:13.724582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:13.725072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.725072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:13.725072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.725072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:13.725072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:13.725072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:13.725072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:13.725072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:13.725072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.725072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:13.725072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:13.725072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.725072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:13.725072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:13.725072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.725635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:13.725635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:13.725635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.725635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:34:13.725635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:13.725635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:13.725635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:13.725635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:13.725635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:13.725635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:13.725635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:13.725635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:13.725635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:13.725635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:13.725635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:13.725635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:13.725635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:13.725635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:13.725635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:13.726287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:13.726287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:13.726287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:13.726287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:13.726287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.726287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:13.919846 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:13.920403 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:13.920622 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:13.922762 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:13.922762 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:13.922762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.923855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:13.923855 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:13.923855 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:13.923855 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:13.923855 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:13.923855 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:13.923855 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.923855 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:13.925093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.925093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:13.925093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:13.925093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:13.925093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.925093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:34:13.925093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:13.925093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:13.925093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:13.925093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:13.925093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:13.925093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:13.925527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:13.925527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:13.925527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:13.925527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:13.925527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:13.925527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:13.925527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:13.925527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:13.925527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:13.925527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:13.925527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:13.925527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:13.925527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:13.925962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.925962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:13.926197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.926197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:13.926197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:13.926197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:13.926197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:13.926197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:13.926197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.926197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:13.926197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:13.926197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.926197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:13.926197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:13.926197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.926197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:13.926197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:13.926197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.926718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:34:13.926718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:13.926718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:13.926718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:13.926718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:13.926718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:13.926718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:13.926718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:13.926718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:13.926718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:13.926718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:13.926718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:13.926718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:13.926718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:13.926718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:13.926718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:13.926718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:13.926718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:13.926718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:13.927272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:13.927272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.927272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:13.967406 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:13.967834 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:13.967956 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:13.968080 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:34:13.970165 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:13.970165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.970362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:13.970362 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:13.970362 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:13.970362 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:13.970362 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:13.970362 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:13.970362 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.970362 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:13.971423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.971423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:34:13.971423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:13.971423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:13.971423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:13.971423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:13.971423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:13.971423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:13.971423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:13.971423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:13.971423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:13.971423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:34:13.971423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:13.971423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:34:13.971423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:34:13.972233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:13.972233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:34:13.972233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:13.972233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:34:13.972233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:34:13.972233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:13.972233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:34:13.972233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.972233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:34:13.972880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.972880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:13.972880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:13.972880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:13.972880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.972880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:34:13.972880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:34:13.972880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:34:13.972880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:13.972880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:13.972880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:13.972880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:13.973581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:13.973581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:13.973581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:13.973581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:13.973581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:13.973581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:13.973581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:13.973581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:13.973581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:13.973581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:13.973581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:13.973581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:13.973581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:13.973581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:13.974241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:13.974241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:13.974241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.974241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:13.974241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.974241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:13.974241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:13.974241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:13.974241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:13.974241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:13.974241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.974241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:13.974241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:13.974241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.974850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:13.974850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:13.974850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.974850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:13.974850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:13.974850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.974850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:34:13.974850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:34:13.974850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:34:13.974850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:13.974850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:13.974850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:13.974850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:13.974850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:13.974850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:13.974850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:13.974850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:13.974850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:13.974850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:13.975726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:13.975726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:13.975726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:13.975726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:13.975726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:13.975726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:13.975726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:13.975726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:13.975726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:13.975726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:13.975726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.975726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:13.976223 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:13.976763 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:13.976763 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:13.978873 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:13.978873 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:13.978873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.978873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:13.978873 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:13.978873 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:13.978873 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:13.978873 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:13.978873 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:13.978873 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.978873 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:13.979541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.979541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:34:13.979541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:13.979541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:13.979541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:13.979541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:13.979541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:13.979541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:13.979541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:13.979541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:13.979541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:13.979541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:34:13.979541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:13.979541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:34:13.979541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:34:13.980463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:13.980463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:34:13.980463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:13.980463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:34:13.980463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:34:13.980463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:13.980463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:34:13.980463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.980463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:34:13.980463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.980463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:13.980463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:13.980463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:13.980463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:34:13.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:34:13.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:13.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:13.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:13.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:13.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:13.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:13.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:13.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:13.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:13.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:13.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:13.981915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:13.981915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:13.981915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:13.981915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:13.981915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:13.981915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:13.981915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:13.981915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:13.981915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:13.981915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.981915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:13.981915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.982293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:13.982293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:13.982293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:13.982293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:13.982293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:13.982293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.982293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:13.982293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:13.982293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.982293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:13.982293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:13.982293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.982293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:13.982293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:13.982293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.982293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:34:13.982293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:34:13.982293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:13.982293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:13.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:13.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:13.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:13.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:13.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:13.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:13.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:13.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:13.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:13.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:13.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:13.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:13.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:13.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:13.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:13.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:13.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:13.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:13.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.982855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:13.984340 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:13.984659 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:13.984790 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:13.984946 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:13.985979 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:13.985979 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:13.985979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.986119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:13.986119 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:13.986119 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:13.986119 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:13.986119 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:13.986119 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:13.986119 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.986119 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:13.986869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.986869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:13.986869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:13.986869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:13.986869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.986869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:34:13.986869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:13.986869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:13.986869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:13.986869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:13.986869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:13.986869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:13.987267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:13.987267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:13.987267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:13.987267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:13.987267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:13.987267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:13.987267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:13.987267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:13.987267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:13.987267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:13.987267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:13.987267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:13.987267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:13.987651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.987651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:13.987944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.987944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:13.987944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:13.987944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:13.987944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:13.987944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:13.987944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.987944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:13.987944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:13.987944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.987944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:13.987944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:13.987944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.987944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:13.987944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:13.987944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:13.988412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:34:13.988412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:13.988412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:13.988412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:13.988412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:13.988412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:13.988412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:13.988412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:13.988412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:13.988412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:13.988412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:13.988412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:13.988412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:13.988412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:13.988412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:13.988412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:13.988412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:13.988412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:13.988412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:13.989514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:13.989514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:13.989514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:18.687450 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:18.688202 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:18.688202 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:18.688795 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:18.724506 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:18.724506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.724800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:18.724800 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:18.724800 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:18.724800 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:18.724800 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:18.724800 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:18.724800 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.724800 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:18.725560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.725560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:18.725560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:18.725560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:18.725560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.725560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:34:18.725560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:18.725560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:18.725560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:18.725560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:18.725560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:18.725560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:18.726150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:18.726150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:18.726150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:18.726150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:18.726150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:18.726150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:18.726150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:18.726150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:18.726150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:18.726150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:18.726150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:18.726150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:18.726150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:18.726648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.726648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:18.726648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.726648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:18.726648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:18.726648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:18.726648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:18.726648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:18.726648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.726648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:18.726648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:18.726648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.726648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:18.726648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:18.726648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.727313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:18.727313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:18.727313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.727313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:34:18.727313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:18.727313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:18.727313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:18.727313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:18.727313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:18.727313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:18.727313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:18.727313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:18.727313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:18.727313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:18.727313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:18.727313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:18.727313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:18.727313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:18.727313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:18.728085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:18.728085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:18.728085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:18.728085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:18.728085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.728085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:18.920944 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:18.921107 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:18.921172 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:18.922047 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:18.922429 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:18.922660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.924492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:18.924492 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:18.924492 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:18.924492 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:18.924492 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:18.924492 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:18.924492 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.924492 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:18.925619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.925619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:18.925619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:18.925619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:18.925619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.925619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:34:18.925619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:18.925619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:18.925619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:18.925619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:18.925619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:18.925619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:18.926544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:18.926544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:18.926544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:18.926544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:18.926544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:18.926544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:18.926544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:18.926544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:18.926544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:18.926544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:18.926544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:18.926544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:18.926544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:18.927486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.927486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:18.927486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.927486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:18.927486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:18.927486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:18.927486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:18.927486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:18.927486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.927486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:18.927486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:18.927486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.927486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:18.927486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:18.927486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.928543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:18.928543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:18.928543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.928543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:34:18.928543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:18.928543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:18.928543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:18.928543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:18.928543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:18.928543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:18.928543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:18.928543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:18.928543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:18.928543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:18.928543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:18.928543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:18.928543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:18.928543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:18.928543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:18.929622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:18.929622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:18.929622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:18.929622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:18.929622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.929622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:18.967933 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:18.968280 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:18.968393 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:18.968512 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:34:18.969110 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:18.969533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.970428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:18.970428 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:18.970428 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:18.970428 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:18.970428 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:18.970428 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:18.970428 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.970428 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:18.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:34:18.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:18.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:18.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:18.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:18.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:18.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:18.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:18.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:18.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:18.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:34:18.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:18.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:34:18.971211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:34:18.971657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:18.971657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:34:18.971657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:18.971657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:34:18.971657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:34:18.971657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:18.971657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:34:18.971657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.971657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:34:18.972042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.972042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:18.972042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:18.972042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:18.972042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.972042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:34:18.972042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:34:18.972042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:34:18.972042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:18.972042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:18.972042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:18.972042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:18.972416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:18.972416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:18.972416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:18.972416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:18.972416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:18.972416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:18.972416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:18.972416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:18.972416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:18.972416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:18.972416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:18.972416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:18.972416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:18.972416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:18.972805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:18.972805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:18.972805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.972805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:18.973109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.973109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:18.973109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:18.973109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:18.973109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:18.973109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:18.973109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.973109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:18.973109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:18.973109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.973109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:18.973109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:18.973109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.973109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:18.973109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:18.973109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.973565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:34:18.973565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:34:18.973565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:34:18.973565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:18.973565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:18.973565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:18.973565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:18.973565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:18.973565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:18.973565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:18.973565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:18.973565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:18.973565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:18.973565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:18.973565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:18.973565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:18.973565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:18.973565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:18.974108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:18.974108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:18.974108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:18.974108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:18.974108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:18.974108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.974108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:18.976693 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:18.977058 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:18.977154 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:18.977301 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:18.977758 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:18.978116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.978699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:18.978699 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:18.978699 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:18.978699 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:18.978699 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:18.978699 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:18.978699 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.978699 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:18.979768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.979768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:34:18.979768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:18.979768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:18.979768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:18.979768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:18.979768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:18.979768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:18.979768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:18.979768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:18.979768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:18.979768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:34:18.979768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:18.979768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:34:18.979768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:34:18.980358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:18.980358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:34:18.980358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:18.980358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:34:18.980358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:34:18.980358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:18.980358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:34:18.980358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.980358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:34:18.981032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.981032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:18.981032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:18.981032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:18.981032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.981032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:34:18.981032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:34:18.981032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:18.981032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:18.981032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:18.981032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:18.981390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:18.981390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:18.981390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:18.981390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:18.981390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:18.981390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:18.981390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:18.981390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:18.981390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:18.981390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:18.981390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:18.981390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:18.981390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:18.981390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:18.981771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:18.981771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:18.981771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.981771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:18.982560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.982560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:18.982560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:18.982560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:18.982560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:18.982560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:18.982560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.982560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:18.982560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:18.982560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.982560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:18.982560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:18.982560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.982560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:18.982560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:18.982560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.982560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:34:18.983035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:34:18.983035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:18.983035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:18.983035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:18.983035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:18.983035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:18.983035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:18.983035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:18.983035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:18.983035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:18.983035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:18.983035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:18.983035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:18.983035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:18.983035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:18.983035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:18.983035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:18.983035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:18.983035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:18.983501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:18.983501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:18.983501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.983501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:18.984766 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:18.985091 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:18.985242 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:18.985442 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:18.985789 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:18.986051 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:18.986202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.987098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:18.987098 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:18.987098 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:18.987098 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:18.987098 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:18.987098 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:18.987098 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.987098 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:18.988963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.988963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:18.988963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:18.988963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:18.988963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.988963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:34:18.988963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:18.988963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:18.988963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:18.988963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:18.988963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:18.988963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:18.989660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:18.989660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:18.989660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:18.989660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:18.989660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:18.989660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:18.989660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:18.989660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:18.989660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:18.989660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:18.989660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:18.989660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:18.989660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:18.990061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.990061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:18.991119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.991119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:18.991119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:18.991119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:18.991119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:18.991119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:18.991119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.991119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:18.991119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:18.991119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.991119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:18.991119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:18.991119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.991119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:18.991119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:18.991119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:18.991670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:34:18.991670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:18.991670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:18.991670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:18.991670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:18.991670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:18.991670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:18.991670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:18.991670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:18.991670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:18.991670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:18.991670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:18.991670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:18.991670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:18.991670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:18.991670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:18.991670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:18.991670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:18.991670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:18.992305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:18.992305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:18.992305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:23.688033 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:23.689204 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:23.689204 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:23.690122 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:23.723168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:23.723650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.724528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:23.724528 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:23.724528 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:23.724528 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:23.724528 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:23.724528 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:23.724528 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.724528 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:23.725338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.725338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:23.725338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:23.725338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:23.725338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.725338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:34:23.725338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:23.725338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:23.725338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:23.725338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:23.725338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:23.725338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:23.726415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:23.726415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:23.726415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:23.726415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:23.726415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:23.726415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:23.726415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:23.726415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:23.726415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:23.726415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:23.726415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:23.726415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:23.726415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:23.727085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.727085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:23.727085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.727085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:23.727085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:23.727085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:23.727085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:23.727085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:23.727085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.727085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:23.727085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:23.727085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.727085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:23.727085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:23.727085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.727879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:23.727879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:23.727879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.727879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:34:23.727879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:23.727879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:23.727879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:23.727879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:23.727879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:23.727879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:23.727879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:23.727879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:23.727879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:23.727879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:23.727879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:23.727879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:23.727879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:23.727879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:23.727879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:23.728892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:23.728892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:23.728892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:23.728892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:23.728892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.728892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:23.921123 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:23.921794 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:23.922057 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:23.923119 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:23.923461 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:23.923762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.924663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:23.924663 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:23.924663 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:23.924663 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:23.924663 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:23.924663 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:23.924663 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.924663 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:23.926476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.926476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:23.926476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:23.926476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:23.926476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.926476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:34:23.926476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:23.926476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:23.926476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:23.926476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:23.926476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:23.926476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:23.927270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:23.927270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:23.927270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:23.927270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:23.927270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:23.927270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:23.927270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:23.927270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:23.927270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:23.927270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:23.927270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:23.927270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:23.927270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:23.927914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.927914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:23.928301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.928301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:23.928301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:23.928301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:23.928301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:23.928301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:23.928301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.928301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:23.928301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:23.928301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.928301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:23.928301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:23.928301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.928301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:23.928301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:23.928301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.928818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:34:23.928818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:23.928818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:23.928818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:23.928818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:23.928818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:23.928818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:23.928818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:23.928818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:23.928818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:23.928818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:23.928818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:23.928818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:23.928818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:23.928818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:23.928818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:23.928818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:23.928818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:23.928818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:23.929433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:23.929433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.929433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:23.968462 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:23.968960 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:23.969157 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:23.969350 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:34:23.970292 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:23.970889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.971945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:23.971945 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:23.971945 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:23.971945 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:23.971945 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:23.971945 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:23.971945 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.971945 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:23.973281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.973281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:34:23.973281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:23.973281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:23.973281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:23.973281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:23.973281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:23.973281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:23.973281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:23.973281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:23.973281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:23.973281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:34:23.973281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:23.973281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:34:23.973281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:34:23.973816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:23.973816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:34:23.973816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:23.973816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:34:23.973816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:34:23.973816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:23.973816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:34:23.973816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.973816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:34:23.974637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.974637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:23.974637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:23.974637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:23.974637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.974637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:34:23.974637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:34:23.974637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:34:23.974637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:23.974637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:23.974637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:23.974637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:23.975203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:23.975203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:23.975203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:23.975203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:23.975203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:23.975203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:23.975203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:23.975203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:23.975203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:23.975203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:23.975203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:23.975203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:23.975203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:23.975203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:23.975630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:23.975630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:23.975630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.975630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:23.976753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.976753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:23.976753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:23.976753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:23.976753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:23.976753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:23.976753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.976753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:23.976753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:23.976753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.976753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:23.976753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:23.976753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.976753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:23.976753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:23.976753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.977512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:34:23.977512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:34:23.977512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:34:23.977512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:23.977512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:23.977512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:23.977512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:23.977512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:23.977512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:23.977512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:23.977512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:23.977512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:23.977512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:23.977512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:23.977512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:23.977512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:23.977512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:23.977512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:23.978741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:23.978741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:23.978741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:23.978741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:23.978741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:23.978741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.978741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:23.978741 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:23.978741 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:23.978741 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:23.979194 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:23.979194 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:23.979194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.979305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:23.979305 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:23.979305 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:23.979305 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:23.979305 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:23.979305 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:23.979305 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.979305 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:23.980053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.980053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:34:23.980053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:23.980053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:23.980053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:23.980053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:23.980053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:23.980053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:23.980053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:23.980053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:23.980053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:23.980053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:34:23.980053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:23.980053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:34:23.980053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:34:23.980502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:23.980502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:34:23.980502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:23.980502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:34:23.980502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:34:23.980502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:23.980502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:34:23.980502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.980502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:34:23.980929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.980929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:23.980929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:23.980929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:23.980929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.980929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:34:23.980929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:34:23.980929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:23.980929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:23.980929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:23.980929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:23.981211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:23.981211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:23.981211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:23.981211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:23.981211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:23.981211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:23.981211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:23.981211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:23.981211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:23.981211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:23.981211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:23.981211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:23.981211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:23.981211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:23.981568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:23.981568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:23.981568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.981568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:23.982010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.982010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:23.982010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:23.982010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:23.982010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:23.982010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:23.982010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.982010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:23.982010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:23.982010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.982010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:23.982010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:23.982010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.982010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:23.982010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:23.982010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.982010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:34:23.982403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:34:23.982403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:23.982403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:23.982403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:23.982403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:23.982403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:23.982403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:23.982403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:23.982403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:23.982403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:23.982403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:23.982403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:23.982403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:23.982403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:23.982403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:23.982403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:23.982403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:23.982403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:23.982403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:23.982803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:23.982803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:23.982803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.982803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:23.985144 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:23.985466 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:23.985601 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:23.985768 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:23.986092 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:23.986384 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:23.986472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.987180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:23.987180 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:23.987180 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:23.987180 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:23.987180 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:23.987180 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:23.987180 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.987180 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:23.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:23.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:23.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:23.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:34:23.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:23.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:23.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:23.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:23.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:23.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:23.989029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:23.989029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:23.989029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:23.989029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:23.989029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:23.989029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:23.989029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:23.989029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:23.989029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:23.989029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:23.989029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:23.989029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:23.989029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:23.989635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.989635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:23.989635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.989635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:23.989635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:23.989635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:23.989635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:23.989635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:23.989635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.989635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:23.989635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:23.989635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.989635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:23.989635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:23.989635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.990387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:23.990387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:23.990387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:23.990387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:34:23.990387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:23.990387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:23.990387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:23.990387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:23.990387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:23.990387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:23.990387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:23.990387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:23.990387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:23.990387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:23.990387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:23.990387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:23.990387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:23.990387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:23.990387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:23.991309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:23.991309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:23.991309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:23.991309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:23.991309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:23.991309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:28.688757 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:28.689529 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:28.689529 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:28.690595 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:28.728476 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:28.728966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.729791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:28.729791 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:28.729791 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:28.729791 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:28.729791 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:28.729791 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:28.729791 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.729791 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:28.730698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.730698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:28.730698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:28.730698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:28.730698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.730698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:34:28.730698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:28.730698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:28.730698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:28.730698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:28.730698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:28.730698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:28.731293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:28.731293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:28.731293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:28.731293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:28.731293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:28.731293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:28.731293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:28.731293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:28.731293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:28.731293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:28.731293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:28.731293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:28.731293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:28.731907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.731907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:28.731907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.731907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:28.731907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:28.731907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:28.731907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:28.731907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:28.731907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.731907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:28.731907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:28.731907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.731907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:28.731907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:28.731907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.732716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:28.732716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:28.732716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.732716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:34:28.732716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:28.732716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:28.732716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:28.732716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:28.732716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:28.732716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:28.732716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:28.732716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:28.732716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:28.732716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:28.732716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:28.732716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:28.732716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:28.732716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:28.732716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:28.733646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:28.733646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:28.733646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:28.733646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:28.733646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.733646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:28.921882 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:28.922511 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:28.922702 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:28.923746 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:28.924058 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:28.924373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.925274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:28.925274 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:28.925274 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:28.925274 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:28.925274 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:28.925274 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:28.925274 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.925274 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:28.926829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.926829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:28.926829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:28.926829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:28.926829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.926829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:34:28.926829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:28.926829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:28.926829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:28.926829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:28.926829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:28.926829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:28.927418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:28.927418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:28.927418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:28.927418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:28.927418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:28.927418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:28.927418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:28.927418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:28.927418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:28.927418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:28.927418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:28.927418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:28.927418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:28.928037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.928037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:28.928523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.928523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:28.928523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:28.928523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:28.928523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:28.928523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:28.928523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.928523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:28.928523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:28.928523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.928523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:28.928523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:28.928523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.928523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:28.928523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:28.928523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.929425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:34:28.929425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:28.929425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:28.929425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:28.929425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:28.929425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:28.929425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:28.929425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:28.929425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:28.929425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:28.929425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:28.929425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:28.929425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:28.929425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:28.929425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:28.929425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:28.929425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:28.929425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:28.929425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:28.930351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:28.930351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.930351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:28.969007 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:28.969425 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:28.969608 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:28.969842 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:34:28.970444 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:28.970948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.971544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:28.971544 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:28.971544 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:28.971544 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:28.971544 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:28.971544 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:28.971544 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.971544 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:28.972885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.972885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:34:28.972885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:28.972885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:28.972885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:28.972885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:28.972885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:28.972885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:28.972885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:28.972885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:28.972885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:28.972885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:34:28.972885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:28.972885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:34:28.972885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:34:28.973389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:28.973389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:34:28.973389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:28.973389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:34:28.973389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:34:28.973389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:28.973389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:34:28.973389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.973389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:34:28.974887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.974887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:28.974887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:28.974887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:28.974887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.974887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:34:28.974887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:34:28.974887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:34:28.974887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:28.974887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:28.974887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:28.974887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:28.975280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:28.975280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:28.975280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:28.975280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:28.975280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:28.975280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:28.975280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:28.975280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:28.975280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:28.975280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:28.975280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:28.975280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:28.975280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:28.975280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:28.975775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:28.975775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:28.975775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.975775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:28.976179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.976179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:28.976179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:28.976179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:28.976179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:28.976179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:28.976179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.976179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:28.976179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:28.976179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.976179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:28.976179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:28.976179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.976179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:28.976179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:28.976179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.976626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:34:28.976626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:34:28.976626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:34:28.976626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:28.976626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:28.976626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:28.976626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:28.976626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:28.976626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:28.976626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:28.976626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:28.976626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:28.976626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:28.976626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:28.976626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:28.976626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:28.976626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:28.976626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:28.977163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:28.977163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:28.977163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:28.977163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:28.977163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:28.977163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.977163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:28.977655 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:28.977954 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:28.978165 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:28.978395 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:28.978781 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:28.979051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.979809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:28.979809 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:28.979809 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:28.979809 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:28.979809 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:28.979809 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:28.979809 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.979809 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:28.980690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.980690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:34:28.980690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:28.980690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:28.980690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:28.980690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:28.980690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:28.980690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:28.980690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:28.980690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:28.980690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:28.980690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:34:28.980690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:28.980690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:34:28.980690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:34:28.981120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:28.981120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:34:28.981120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:28.981120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:34:28.981120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:34:28.981120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:28.981120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:34:28.981120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.981120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:34:28.981619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.981619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:28.981619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:28.981619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:28.981619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.981619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:34:28.981619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:34:28.981619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:28.981619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:28.981619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:28.981619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:28.981982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:28.981982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:28.981982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:28.981982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:28.981982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:28.981982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:28.981982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:28.981982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:28.981982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:28.981982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:28.981982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:28.981982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:28.981982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:28.981982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:28.982408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:28.982408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:28.982408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.982408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:28.983005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.983005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:28.983005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:28.983005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:28.983005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:28.983005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:28.983005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.983005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:28.983005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:28.983005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.983005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:28.983005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:28.983005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.983005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:28.983005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:28.983005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.983005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:34:28.983513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:34:28.983513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:28.983513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:28.983513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:28.983513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:28.983513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:28.983513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:28.983513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:28.983513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:28.983513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:28.983513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:28.983513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:28.983513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:28.983513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:28.983513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:28.983513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:28.983513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:28.983513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:28.983513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:28.984287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:28.984287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:28.984287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.984287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:28.985489 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:28.985776 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:28.985863 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:28.986195 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:28.986480 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:28.986790 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:28.986970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.987568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:28.987568 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:28.987568 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:28.987568 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:28.987568 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:28.987568 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:28.987568 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.987568 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:28.988403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.988403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:28.988403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:28.988403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:28.988403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.988403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:34:28.988403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:28.988403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:28.988403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:28.988403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:28.988403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:28.988403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:28.988815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:28.988815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:28.988815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:28.988815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:28.988815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:28.988815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:28.988815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:28.988815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:28.988815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:28.988815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:28.988815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:28.988815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:28.988815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:28.989146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.989146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:28.989365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.989365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:28.989365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:28.989365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:28.989365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:28.989365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:28.989365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.989365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:28.989365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:28.989365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.989365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:28.989365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:28.989365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.989365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:28.989365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:28.989365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:28.989922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:34:28.989922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:28.989922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:28.989922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:28.989922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:28.989922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:28.989922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:28.989922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:28.989922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:28.989922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:28.989922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:28.989922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:28.989922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:28.989922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:28.989922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:28.989922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:28.989922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:28.989922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:28.989922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:28.990471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:28.990471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:28.990471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:33.689334 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:33.690523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:33.690523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:33.691183 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:33.722420 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:33.722714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.723344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:33.723344 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:33.723344 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:33.723344 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:33.723344 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:33.723344 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:33.723344 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.723344 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:33.724133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.724133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:33.724133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:33.724133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:33.724133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.724133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:34:33.724133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:33.724133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:33.724133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:33.724133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:33.724133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:33.724133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:33.724946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:33.724946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:33.724946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:33.724946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:33.724946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:33.724946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:33.724946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:33.724946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:33.724946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:33.724946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:33.724946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:33.724946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:33.724946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:33.725649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.725649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:33.725649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.725649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:33.725649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:33.725649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:33.725649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:33.725649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:33.725649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.725649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:33.725649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:33.725649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.725649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:33.725649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:33.725649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.726482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:33.726482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:33.726482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.726482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:34:33.726482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:33.726482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:33.726482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:33.726482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:33.726482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:33.726482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:33.726482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:33.726482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:33.726482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:33.726482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:33.726482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:33.726482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:33.726482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:33.726482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:33.726482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:33.727457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:33.727457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:33.727457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:33.727457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:33.727457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.727457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:33.922399 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:33.922958 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:33.923216 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:33.924747 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:33.925097 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:33.926102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.927521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:33.927521 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:33.927521 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:33.927521 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:33.927521 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:33.927521 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:33.927521 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.927521 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:33.929096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.929096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:33.929096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:33.929096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:33.929096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.929096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:34:33.929096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:33.929096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:33.929096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:33.929096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:33.929096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:33.929096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:33.929538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:33.929538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:33.929538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:33.929538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:33.929538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:33.929538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:33.929538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:33.929538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:33.929538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:33.929538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:33.929538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:33.929538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:33.929538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:33.929936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.929936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:33.930741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.930741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:33.930741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:33.930741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:33.930741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:33.930741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:33.930741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.930741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:33.930741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:33.930741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.930741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:33.930741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:33.930741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.930741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:33.930741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:33.930741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.931240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:34:33.931240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:33.931240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:33.931240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:33.931240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:33.931240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:33.931240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:33.931240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:33.931240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:33.931240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:33.931240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:33.931240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:33.931240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:33.931240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:33.931240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:33.931240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:33.931240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:33.931240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:33.931240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:33.931807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:33.931807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.931807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:33.969603 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:33.970071 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:33.970226 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:33.970412 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:34:33.971106 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:33.971438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.972312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:33.972312 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:33.972312 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:33.972312 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:33.972312 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:33.972312 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:33.972312 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.972312 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:33.973501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.973501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:34:33.973501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:33.973501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:33.973501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:33.973501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:33.973501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:33.973501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:33.973501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:33.973501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:33.973501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:33.973501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:34:33.973501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:33.973501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:34:33.973501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:34:33.974523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:33.974523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:34:33.974523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:33.974523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:34:33.974523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:34:33.974523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:33.974523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:34:33.974523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.974523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:34:33.975089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.975089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:33.975089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:33.975089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:33.975089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.975089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:34:33.975089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:34:33.975089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:34:33.975089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:33.975089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:33.975089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:33.975089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:33.975712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:33.975712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:33.975712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:33.975712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:33.975712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:33.975712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:33.975712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:33.975712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:33.975712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:33.975712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:33.975712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:33.975712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:33.975712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:33.975712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:33.976368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:33.976368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:33.976368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.976368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:33.976368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.976368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:33.976368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:33.976368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:33.976368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:33.976368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:33.976368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.976368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:33.976368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:33.976368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.977110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:33.977110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:33.977110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.977110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:33.977110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:33.977110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.977110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:34:33.977110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:34:33.977110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:34:33.977110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:33.977110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:33.977110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:33.977110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:33.977110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:33.977110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:33.977110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:33.977110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:33.977110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:33.977110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:33.977888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:33.977888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:33.977888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:33.977888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:33.977888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:33.977888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:33.977888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:33.977888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:33.977888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:33.977888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:33.977888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.977888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:33.978260 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:33.978260 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:33.978352 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:33.978481 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:33.979018 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:33.979384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.980138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:33.980138 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:33.980138 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:33.980138 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:33.980138 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:33.980138 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:33.980138 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.980138 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:33.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:34:33.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:33.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:33.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:33.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:33.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:33.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:33.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:33.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:33.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:33.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:34:33.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:33.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:34:33.980920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:34:33.981359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:33.981359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:34:33.981359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:33.981359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:34:33.981359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:34:33.981359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:33.981359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:34:33.981359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.981359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:34:33.981830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.981830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:33.981830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:33.981830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:33.981830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.981830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:34:33.981830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:34:33.981830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:33.981830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:33.981830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:33.981830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:33.982166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:33.982166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:33.982166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:33.982166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:33.982166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:33.982166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:33.982166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:33.982166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:33.982166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:33.982166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:33.982166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:33.982166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:33.982166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:33.982166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:33.982547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:33.982547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:33.982547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.982547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:33.982859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.982859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:33.982859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:33.982859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:33.982859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:33.982859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:33.982859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.982859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:33.982859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:33.982859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.982859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:33.982859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:33.982859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.982859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:33.982859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:33.982859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.982859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:34:33.983373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:34:33.983373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:33.983373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:33.983373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:33.983373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:33.983373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:33.983373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:33.983373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:33.983373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:33.983373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:33.983373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:33.983373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:33.983373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:33.983373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:33.983373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:33.983373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:33.983373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:33.983373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:33.983373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:33.983951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:33.983951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:33.983951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.983951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:33.985923 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:33.986204 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:33.986387 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:33.986450 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:33.986988 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:33.987354 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:33.987592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.988302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:33.988302 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:33.988302 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:33.988302 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:33.988302 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:33.988302 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:33.988302 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.988302 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:33.989147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.989147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:33.989147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:33.989147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:33.989147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.989147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:34:33.989147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:33.989147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:33.989147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:33.989147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:33.989147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:33.989147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:33.989529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:33.989529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:33.989529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:33.989529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:33.989529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:33.989529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:33.989529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:33.989529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:33.989529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:33.989529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:33.989529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:33.989529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:33.989529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:33.989892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.989892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:33.990121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.990121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:33.990121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:33.990121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:33.990121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:33.990121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:33.990121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.990121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:33.990121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:33.990121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.990121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:33.990121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:33.990121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.990121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:33.990121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:33.990121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:33.990587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:34:33.990587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:33.990587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:33.990587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:33.990587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:33.990587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:33.990587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:33.990587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:33.990587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:33.990587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:33.990587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:33.990587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:33.990587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:33.990587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:33.990587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:33.990587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:33.990587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:33.990587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:33.990587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:33.991098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:33.991098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:33.991098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:38.689875 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:38.690642 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:38.690642 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:38.691270 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:38.728583 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:38.728976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.729752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:38.729752 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:38.729752 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:38.729752 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:38.729752 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:38.729752 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:38.729752 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.729752 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:38.730565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.730565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:38.730565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:38.730565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:38.730565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.730565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:34:38.730565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:38.730565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:38.730565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:38.730565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:38.730565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:38.730565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:38.731290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:38.731290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:38.731290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:38.731290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:38.731290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:38.731290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:38.731290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:38.731290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:38.731290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:38.731290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:38.731290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:38.731290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:38.731290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:38.731765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.731765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:38.731765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.731765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:38.731765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:38.731765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:38.731765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:38.731765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:38.731765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.731765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:38.731765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:38.731765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.731765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:38.731765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:38.731765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.732292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:38.732292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:38.732292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.732292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:34:38.732292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:38.732292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:38.732292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:38.732292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:38.732292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:38.732292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:38.732292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:38.732292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:38.732292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:38.732292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:38.732292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:38.732292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:38.732292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:38.732292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:38.732292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:38.732954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:38.732954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:38.732954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:38.732954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:38.732954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.732954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:38.923078 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:38.923621 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:38.923788 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:38.925643 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:38.925643 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:38.925912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.926875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:38.926875 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:38.926875 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:38.926875 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:38.926875 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:38.926875 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:38.926875 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.926875 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:38.929073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.929073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:38.929073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:38.929073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:38.929073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.929073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:34:38.929073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:38.929073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:38.929073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:38.929073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:38.929073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:38.929073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:38.929810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:38.929810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:38.929810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:38.929810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:38.929810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:38.929810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:38.929810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:38.929810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:38.929810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:38.929810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:38.929810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:38.929810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:38.929810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:38.930457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.930457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:38.931709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.931709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:38.931709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:38.931709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:38.931709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:38.931709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:38.931709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.931709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:38.931709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:38.931709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.931709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:38.931709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:38.931709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.931709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:38.931709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:38.931709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.932547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:34:38.932547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:38.932547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:38.932547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:38.932547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:38.932547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:38.932547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:38.932547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:38.932547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:38.932547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:38.932547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:38.932547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:38.932547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:38.932547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:38.932547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:38.932547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:38.932547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:38.932547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:38.932547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:38.933505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:38.933505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.933505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:38.970111 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:38.971026 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:38.971261 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:38.971526 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:34:38.972238 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:38.972513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.973318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:38.973318 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:38.973318 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:38.973318 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:38.973318 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:38.973318 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:38.973318 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.973318 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:38.974849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.974849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:34:38.974849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:38.974849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:38.974849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:38.974849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:38.974849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:38.974849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:38.974849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:38.974849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:38.974849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:38.974849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:34:38.974849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:38.974849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:34:38.974849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:34:38.975319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:38.975319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:34:38.975319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:38.975319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:34:38.975319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:34:38.975319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:38.975319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:34:38.975319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.975319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:34:38.976438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.976438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:38.976438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:38.976438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:38.976438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.976438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:34:38.976438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:34:38.976438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:34:38.976438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:38.976438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:38.976438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:38.976438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:38.976829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:38.976829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:38.976829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:38.976829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:38.976829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:38.976829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:38.976829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:38.976829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:38.976829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:38.976829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:38.976829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:38.976829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:38.976829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:38.976829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:38.977300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:38.977300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:38.977300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.977300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:38.977638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.977638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:38.977638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:38.977638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:38.977638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:38.977638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:38.977638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.977638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:38.977638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:38.977638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.977638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:38.977638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:38.977638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.977638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:38.977638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:38.977638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.978151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:34:38.978151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:34:38.978151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:34:38.978151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:38.978151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:38.978151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:38.978151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:38.978151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:38.978151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:38.978151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:38.978151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:38.978151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:38.978151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:38.978151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:38.978151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:38.978151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:38.978151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:38.978151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:38.978665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:38.978665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:38.978665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:38.978665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:38.978665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:38.978665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.978665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:38.978665 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:38.978950 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:38.978950 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:38.979109 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:38.979775 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:38.980094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.980782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:38.980782 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:38.980782 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:38.980782 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:38.980782 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:38.980782 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:38.980782 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.980782 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:38.981709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.981709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:34:38.981709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:38.981709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:38.981709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:38.981709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:38.981709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:38.981709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:38.981709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:38.981709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:38.981709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:38.981709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:34:38.981709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:38.981709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:34:38.981709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:34:38.982292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:38.982292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:34:38.982292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:38.982292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:34:38.982292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:34:38.982292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:38.982292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:34:38.982292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.982292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:34:38.983183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.983183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:38.983183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:38.983183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:38.983183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.983183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:34:38.983183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:34:38.983183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:38.983183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:38.983183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:38.983183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:38.983519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:38.983519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:38.983519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:38.983519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:38.983519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:38.983519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:38.983519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:38.983519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:38.983519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:38.983519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:38.983519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:38.983519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:38.983519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:38.983519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:38.983993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:38.983993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:38.983993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.983993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:38.984795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.984795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:38.984795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:38.984795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:38.984795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:38.984795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:38.984795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.984795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:38.984795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:38.984795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.984795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:38.984795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:38.984795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.984795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:38.984795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:38.984795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.984795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:34:38.985247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:34:38.985247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:38.985247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:38.985247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:38.985247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:38.985247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:38.985247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:38.985247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:38.985247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:38.985247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:38.985247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:38.985247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:38.985247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:38.985247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:38.985247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:38.985247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:38.985247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:38.985247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:38.985247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:38.985760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:38.985760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:38.985760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.985760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:38.986454 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:38.986858 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:38.986973 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:38.987132 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:38.987549 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:38.987855 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:38.988050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.988633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:38.988633 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:38.988633 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:38.988633 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:38.988633 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:38.988633 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:38.988633 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.988633 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:38.989512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.989512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:38.989512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:38.989512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:38.989512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.989512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:34:38.989512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:38.989512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:38.989512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:38.989512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:38.989512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:38.989512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:38.989912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:38.989912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:38.989912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:38.989912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:38.989912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:38.989912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:38.989912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:38.989912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:38.989912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:38.989912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:38.989912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:38.989912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:38.989912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:38.990245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.990245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:38.990616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.990616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:38.990616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:38.990616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:38.990616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:38.990616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:38.990616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.990616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:38.990616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:38.990616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.990616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:38.990616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:38.990616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.990616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:38.990616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:38.990616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:38.991055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:34:38.991055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:38.991055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:38.991055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:38.991055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:38.991055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:38.991055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:38.991055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:38.991055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:38.991055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:38.991055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:38.991055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:38.991055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:38.991055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:38.991055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:38.991055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:38.991055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:38.991055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:38.991055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:38.991779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:38.991779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:38.991779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:43.690535 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:43.691782 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:43.691782 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:43.692627 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:43.727584 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:43.727788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.728340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:43.728340 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:43.728340 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:43.728340 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:43.728340 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:43.728340 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:43.728340 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.728340 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:43.729098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.729098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:43.729098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:43.729098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:43.729098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:43.729098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:34:43.729098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:43.729098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:43.729098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:43.729098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:43.729098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:43.729098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:43.729622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:43.729622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:43.729622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:43.729622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:43.729622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:43.729622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:43.729622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:43.729622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:43.729622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:43.729622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:43.729622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:43.729622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:43.729622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:43.730163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.730163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:43.730163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.730163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:43.730163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:43.730163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:43.730163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:43.730163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:43.730163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:43.730163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:43.730163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:43.730163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:43.730163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:43.730163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:43.730163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:43.730799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:43.730799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:43.730799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:43.730799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:34:43.730799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:43.730799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:43.730799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:43.730799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:43.730799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:43.730799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:43.730799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:43.730799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:43.730799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:43.730799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:43.730799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:43.730799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:43.730799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:43.730799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:43.730799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:43.731494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:43.731494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:43.731494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:43.731494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:43.731494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.731494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:43.924078 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:43.924451 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:43.924629 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:43.925724 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:43.926238 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:43.926552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.927711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:43.927711 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:43.927711 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:43.927711 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:43.927711 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:43.927711 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:43.927711 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.927711 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:43.929269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.929269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:43.929269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:43.929269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:43.929269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:43.929269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:34:43.929269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:43.929269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:43.929269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:43.929269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:43.929269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:43.929269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:43.929917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:43.929917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:43.929917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:43.929917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:43.929917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:43.929917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:43.929917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:43.929917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:43.929917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:43.929917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:43.929917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:43.929917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:43.929917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:43.930548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.930548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:43.931022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.931022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:43.931022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:43.931022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:43.931022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:43.931022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:43.931022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:43.931022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:43.931022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:43.931022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:43.931022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:43.931022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:43.931022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:43.931022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:43.931022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:43.931022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:43.931833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:34:43.931833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:43.931833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:43.931833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:43.931833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:43.931833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:43.931833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:43.931833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:43.931833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:43.931833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:43.931833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:43.931833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:43.931833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:43.931833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:43.931833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:43.931833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:43.931833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:43.931833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:43.931833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:43.932757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:43.932757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.932757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:43.970772 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:43.971224 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:43.971400 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:43.971580 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:34:43.972213 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:43.972501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.973326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:43.973326 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:43.973326 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:43.973326 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:43.973326 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:43.973326 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:43.973326 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.973326 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:43.974534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.974534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:34:43.974534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:43.974534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:43.974534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:43.974534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:43.974534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:43.974534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:43.974534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:43.974534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:43.974534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:43.974534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:34:43.974534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:43.974534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:34:43.974534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:34:43.975393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:43.975393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:34:43.975393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:43.975393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:34:43.975393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:34:43.975393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:43.975393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:34:43.975393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.975393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:34:43.975926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.975926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:43.975926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:43.975926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:43.975926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:43.975926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:34:43.975926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:34:43.975926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:34:43.975926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:43.975926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:43.975926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:43.975926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:43.976526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:43.976526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:43.976526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:43.976526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:43.976526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:43.976526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:43.976526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:43.976526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:43.976526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:43.976526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:43.976526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:43.976526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:43.976526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:43.976526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:43.977154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:43.977154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:43.977154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.977154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:43.977431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.977431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:43.977431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:43.977431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:43.977431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:43.977431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:43.977431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:43.977431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:43.977431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:43.977431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:43.977431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:43.977431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:43.977431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:43.977431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:43.977431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:43.977431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:43.978226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:34:43.978226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:34:43.978226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:34:43.978226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:43.978226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:43.978226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:43.978226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:43.978226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:43.978226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:43.978226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:43.978226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:43.978226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:43.978226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:43.978226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:43.978226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:43.978226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:43.978226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:43.978226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:43.979108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:43.979108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:43.979108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:43.979108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:43.979108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:43.979108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.979108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:43.979108 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:43.979529 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:43.979529 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:43.979529 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:43.979886 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:43.980246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.980904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:43.980904 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:43.980904 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:43.980904 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:43.980904 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:43.980904 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:43.980904 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.980904 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:43.981623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.981623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:34:43.981623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:43.981623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:43.981623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:43.981623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:43.981623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:43.981623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:43.981623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:43.981623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:43.981623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:43.981623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:34:43.981623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:43.981623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:34:43.981623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:34:43.982431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:43.982431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:34:43.982431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:43.982431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:34:43.982431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:34:43.982431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:43.982431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:34:43.982431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.982431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:34:43.982944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.982944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:43.982944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:43.982944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:43.982944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:43.982944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:34:43.982944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:34:43.982944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:43.982944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:43.982944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:43.982944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:43.983524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:43.983524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:43.983524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:43.983524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:43.983524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:43.983524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:43.983524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:43.983524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:43.983524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:43.983524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:43.983524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:43.983524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:43.983524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:43.983524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:43.984227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:43.984227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:43.984227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.984227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:43.984227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.984227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:43.984227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:43.984227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:43.984227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:43.984227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:43.984227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:43.984227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:43.984227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:43.984227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:43.984919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:43.984919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:43.984919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:43.984919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:43.984919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:43.984919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:43.984919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:34:43.984919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:34:43.984919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:43.984919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:43.984919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:43.984919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:43.984919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:43.984919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:43.984919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:43.984919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:43.984919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:43.984919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:43.984919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:43.985825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:43.985825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:43.985825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:43.985825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:43.985825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:43.985825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:43.985825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:43.985825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:43.985825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:43.985825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:43.985825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:43.986960 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:43.987263 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:43.987376 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:43.987519 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:44.016114 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:44.016114 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:44.016114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:44.016418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:44.016418 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:44.016418 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:44.016418 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:44.016418 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:44.016418 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:44.016418 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:44.016418 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:44.017886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:44.017886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:44.017886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:44.017886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:44.017886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:44.017886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:34:44.017886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:44.017886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:44.017886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:44.017886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:44.017886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:44.017886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:44.018556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:44.018556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:44.018556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:44.018556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:44.018556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:44.018556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:44.018556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:44.018556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:44.018556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:44.018556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:44.018556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:44.018556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:44.018556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:44.019146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:44.019146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:44.019146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:44.019146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:44.019146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:44.019146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:44.019146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:44.019146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:44.019146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:44.019146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:44.019146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:44.019146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:44.019146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:44.019146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:44.019146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:44.019668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:44.019668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:44.019668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:44.019668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:34:44.019668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:44.019668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:44.019668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:44.019668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:44.019668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:44.019668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:44.019668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:44.019668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:44.019668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:44.019668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:44.019668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:44.019668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:44.019668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:44.019668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:44.019668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:44.020262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:44.020262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:44.020262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:44.020262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:44.020262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:44.020262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:48.691085 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:48.691927 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:48.691927 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:48.693171 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:48.726535 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:48.727124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.727768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:48.727768 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:48.727768 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:48.727768 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:48.727768 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:48.727768 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:48.727768 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.727768 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:48.728427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.728427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:48.728427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:48.728427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:48.728427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.728427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:34:48.728427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:48.728427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:48.728427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:48.728427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:48.728427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:48.728427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:48.729771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:48.729771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:48.729771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:48.729771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:48.729771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:48.729771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:48.729771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:48.729771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:48.729771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:48.729771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:48.729771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:48.729771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:48.729771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:48.730347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.730347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:48.730347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.730347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:48.730347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:48.730347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:48.730347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:48.730347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:48.730347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.730347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:48.730347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:48.730347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.730347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:48.730347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:48.730347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.731072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:48.731072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:48.731072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.731072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:34:48.731072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:48.731072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:48.731072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:48.731072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:48.731072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:48.731072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:48.731072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:48.731072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:48.731072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:48.731072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:48.731072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:48.731072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:48.731072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:48.731072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:48.731072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:48.733430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:48.733430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:48.733430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:48.733430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:48.733430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.733430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:48.924635 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:48.925258 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:48.925461 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:48.926519 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:48.926857 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:48.927090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.928151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:48.928151 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:48.928151 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:48.928151 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:48.928151 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:48.928151 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:48.928151 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.928151 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:48.929773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.929773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:48.929773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:48.929773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:48.929773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.929773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:34:48.929773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:48.929773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:48.929773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:48.929773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:48.929773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:48.929773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:48.931332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:48.931332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:48.931332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:48.931332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:48.931332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:48.931332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:48.931332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:48.931332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:48.931332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:48.931332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:48.931332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:48.931332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:48.931332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:48.931922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.931922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:48.931922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.931922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:48.931922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:48.931922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:48.931922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:48.931922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:48.931922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.931922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:48.931922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:48.931922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.931922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:48.931922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:48.931922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.932710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:48.932710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:48.932710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.932710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:34:48.932710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:48.932710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:48.932710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:48.932710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:48.932710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:48.932710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:48.932710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:48.932710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:48.932710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:48.932710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:48.932710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:48.932710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:48.932710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:48.932710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:48.932710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:48.933396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:48.933396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:48.933396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:48.933396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:48.933396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.933396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:48.971349 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:48.971904 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:48.972151 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:48.972357 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:34:48.972917 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:48.973287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.974108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:48.974108 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:48.974108 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:48.974108 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:48.974108 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:48.974108 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:48.974108 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.974108 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:48.975381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.975381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:34:48.975381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:48.975381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:48.975381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:48.975381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:48.975381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:48.975381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:48.975381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:48.975381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:48.975381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:48.975381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:34:48.975381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:48.975381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:34:48.975381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:34:48.976418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:48.976418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:34:48.976418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:48.976418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:34:48.976418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:34:48.976418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:48.976418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:34:48.976418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.976418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:34:48.979117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.979117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:48.979117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:48.979117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:48.979117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.979117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:34:48.979117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:34:48.979117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:34:48.979117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:48.979117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:48.979117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:48.979117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:48.979875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:48.979875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:48.979875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:48.979875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:48.979875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:48.979875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:48.979875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:48.979875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:48.979875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:48.979875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:48.979875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:48.979875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:48.979875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:48.979875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:48.980507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:48.980507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:48.980507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.980507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:48.980507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.980507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:48.980507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:48.980507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:48.980507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:48.980507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:48.980507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.980507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:48.980507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:48.980507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.981260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:48.981260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:48.981260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.981260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:48.981260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:48.981260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.981260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:34:48.981260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:34:48.981260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:34:48.981260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:48.981260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:48.981260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:48.981260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:48.981260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:48.981260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:48.981260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:48.981260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:48.981260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:48.981260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:48.981797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:48.981797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:48.981797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:48.981797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:48.981797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:48.981797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:48.981797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:48.981797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:48.981797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:48.981797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:48.981797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.981797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:48.981797 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:48.981797 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:48.982181 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:48.982181 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:48.982181 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:48.982181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.982465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:48.982465 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:48.982465 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:48.982465 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:48.982465 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:48.982465 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:48.982465 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.982465 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:48.983174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.983174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:34:48.983174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:48.983174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:48.983174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:48.983174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:48.983174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:48.983174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:48.983174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:48.983174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:48.983174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:48.983174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:34:48.983174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:48.983174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:34:48.983174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:34:48.983634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:48.983634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:34:48.983634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:48.983634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:34:48.983634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:34:48.983634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:48.983634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:34:48.983634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.983634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:34:48.984084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.984084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:48.984084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:48.984084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:48.984084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.984084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:34:48.984084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:34:48.984084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:48.984084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:48.984084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:48.984084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:48.984409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:48.984409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:48.984409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:48.984409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:48.984409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:48.984409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:48.984409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:48.984409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:48.984409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:48.984409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:48.984409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:48.984409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:48.984409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:48.984409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:48.984882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:48.984882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:48.984882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.984882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:48.985505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.985505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:48.985505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:48.985505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:48.985505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:48.985505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:48.985505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.985505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:48.985505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:48.985505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.985505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:48.985505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:48.985505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.985505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:48.985505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:48.985505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.985505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:34:48.986030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:34:48.986030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:48.986030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:48.986030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:48.986030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:48.986030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:48.986030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:48.986030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:48.986030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:48.986030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:48.986030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:48.986030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:48.986030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:48.986030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:48.986030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:48.986030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:48.986030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:48.986030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:48.986030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:48.986498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:48.986498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:48.986498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.986498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:48.987324 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:48.987556 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:48.987662 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:48.987842 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:48.988218 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:48.988622 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:48.988968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.989632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:48.989632 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:48.989632 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:48.989632 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:48.989632 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:48.989632 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:48.989632 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.989632 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:48.990439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.990439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:48.990439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:48.990439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:48.990439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.990439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:34:48.990439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:48.990439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:48.990439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:48.990439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:48.990439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:48.990439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:48.990803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:48.990803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:48.990803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:48.990803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:48.990803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:48.990803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:48.990803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:48.990803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:48.990803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:48.990803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:48.990803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:48.990803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:48.990803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:48.991241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.991241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:48.991450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.991450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:48.991450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:48.991450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:48.991450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:48.991450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:48.991450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.991450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:48.991450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:48.991450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.991450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:48.991450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:48.991450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.991450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:48.991450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:48.991450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:48.991884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:34:48.991884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:48.991884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:48.991884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:48.991884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:48.991884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:48.991884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:48.991884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:48.991884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:48.991884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:48.991884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:48.991884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:48.991884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:48.991884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:48.991884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:48.991884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:48.991884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:48.991884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:48.991884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:48.992427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:48.992427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:48.992427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:53.691754 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:53.692877 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:53.692877 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:53.693777 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:53.728327 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:53.728794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.729605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:53.729605 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:53.729605 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:53.729605 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:53.729605 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:53.729605 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:53.729605 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.729605 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:53.730418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.730418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:53.730418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:53.730418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:53.730418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.730418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:34:53.730418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:53.730418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:53.730418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:53.730418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:53.730418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:53.730418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:53.731205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:53.731205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:53.731205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:53.731205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:53.731205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:53.731205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:53.731205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:53.731205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:53.731205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:53.731205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:53.731205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:53.731205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:53.731205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:53.731952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.731952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:53.731952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.731952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:53.731952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:53.731952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:53.731952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:53.731952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:53.731952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.731952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:53.731952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:53.731952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.731952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:53.731952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:53.731952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.732781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:53.732781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:53.732781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.732781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:34:53.732781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:53.732781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:53.732781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:53.732781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:53.732781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:53.732781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:53.732781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:53.732781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:53.732781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:53.732781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:53.732781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:53.732781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:53.732781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:53.732781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:53.732781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:53.733829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:53.733829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:53.733829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:53.733829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:53.733829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.733829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:53.925450 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:53.926006 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:53.926203 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:53.927094 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:53.927379 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:53.928285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.928285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:53.928285 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:53.928285 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:53.928285 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:53.928285 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:53.928285 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:53.928285 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.928285 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:53.929217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.929217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:53.929217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:53.929217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:53.929217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.929217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:34:53.929217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:53.929217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:53.929217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:53.929217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:53.929217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:53.929217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:53.929828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:53.929828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:53.929828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:53.929828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:53.929828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:53.929828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:53.929828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:53.929828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:53.929828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:53.929828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:53.929828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:53.929828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:53.929828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:53.930399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.930399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:53.930399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.930399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:53.930399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:53.930399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:53.930399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:53.930399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:53.930399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.930399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:53.930399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:53.930399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.930399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:53.930399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:53.930399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.931075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:53.931075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:53.931075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.931075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:34:53.931075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:53.931075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:53.931075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:53.931075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:53.931075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:53.931075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:53.931075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:53.931075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:53.931075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:53.931075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:53.931075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:53.931075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:53.931075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:53.931075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:53.931075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:53.931934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:53.931934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:53.931934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:53.931934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:53.931934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.931934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:53.972711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:53.972711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:53.972711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:53.972996 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:34:53.973479 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:53.973809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.974577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:53.974577 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:53.974577 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:53.974577 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:53.974577 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:53.974577 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:53.974577 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.974577 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:53.976009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.976009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:34:53.976009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:53.976009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:53.976009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:53.976009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:53.976009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:53.976009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:53.976009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:53.976009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:53.976009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:53.976009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:34:53.976009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:53.976009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:34:53.976009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:34:53.976575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:53.976575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:34:53.976575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:53.976575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:34:53.976575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:34:53.976575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:53.976575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:34:53.976575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.976575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:34:53.977326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.977326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:53.977326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:53.977326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:53.977326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.977326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:34:53.977326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:34:53.977326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:34:53.977326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:53.977326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:53.977326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:53.977326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:53.978160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:53.978160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:53.978160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:53.978160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:53.978160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:53.978160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:53.978160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:53.978160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:53.978160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:53.978160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:53.978160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:53.978160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:53.978160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:53.978160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:53.978806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:53.978806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:53.978806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.978806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:53.978994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.978994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:53.978994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:53.978994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:53.978994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:53.978994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:53.978994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.978994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:53.978994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:53.978994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.978994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:53.978994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:53.978994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.978994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:53.978994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:53.978994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.979508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:34:53.979508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:34:53.979508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:34:53.979508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:53.979508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:53.979508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:53.979508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:53.979508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:53.979508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:53.979508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:53.979508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:53.979508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:53.979508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:53.979508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:53.979508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:53.979508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:53.979508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:53.979508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:53.980179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:53.980179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:53.980179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:53.980179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:53.980179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:53.980179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.980179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:53.981925 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:53.981925 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:53.981925 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:53.981925 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:53.982350 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:53.982350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.983340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:53.983340 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:53.983340 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:53.983340 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:53.983340 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:53.983340 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:53.983340 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.983340 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:53.984375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.984375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:34:53.984375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:53.984375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:53.984375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:53.984375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:53.984375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:53.984375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:53.984375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:53.984375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:53.984375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:53.984375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:34:53.984375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:53.984375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:34:53.984375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:34:53.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:53.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:34:53.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:53.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:34:53.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:34:53.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:53.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:34:53.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:34:53.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:53.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:53.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:53.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.985914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:34:53.985914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:34:53.985914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:53.985914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:53.985914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:53.985914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:53.985914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:53.985914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:53.985914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:53.985914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:53.985914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:53.985914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:53.985914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:53.986573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:53.986573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:53.986573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:53.986573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:53.986573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:53.986573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:53.986573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:53.986573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:53.986573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:53.986573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.986573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:53.986573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.987330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:53.987330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:53.987330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:53.987330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:53.987330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:53.987330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.987330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:53.987330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:53.987330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.987330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:53.987330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:53.987330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.987330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:53.987330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:53.987330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.987330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:34:53.987330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:34:53.987330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:53.987330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:53.988071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:53.988071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:53.988071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:53.988071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:53.988071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:53.988071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:53.988071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:53.988071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:53.988071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:53.988071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:53.988071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:53.988071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:53.988071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:53.988071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:53.988071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:53.988071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:53.988071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:53.988071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:53.988071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.988071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:53.988746 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:53.988746 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:53.988746 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:53.988746 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:53.988907 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:53.989210 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:53.989426 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.990144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:53.990144 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:53.990144 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:53.990144 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:53.990144 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:53.990144 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:53.990144 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.990144 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:53.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:53.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:53.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:53.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:34:53.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:53.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:53.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:53.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:53.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:53.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:53.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:53.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:53.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:53.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:53.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:53.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:53.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:53.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:53.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:53.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:53.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:53.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:53.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:53.991903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.991903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:53.992124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.992124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:53.992124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:53.992124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:53.992124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:53.992124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:53.992124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.992124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:53.992124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:53.992124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.992124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:53.992124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:53.992124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.992124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:53.992124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:53.992124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:53.992570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:34:53.992570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:53.992570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:53.992570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:53.992570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:53.992570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:53.992570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:53.992570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:53.992570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:53.992570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:53.992570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:53.992570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:53.992570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:53.992570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:53.992570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:53.992570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:53.992570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:53.992570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:53.992570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:53.993135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:53.993135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:53.993135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:58.694734 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:58.695421 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:58.695421 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:58.698712 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:58.730201 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:58.730580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.731279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:58.731279 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:58.731279 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:58.731279 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:58.731279 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:58.731279 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:58.731279 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.731279 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:58.732335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.732335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:58.732335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:58.732335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:58.732335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.732335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:34:58.732335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:58.732335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:58.732335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:58.732335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:58.732335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:58.732335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:58.732804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:58.732804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:58.732804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:58.732804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:58.732804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:58.732804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:58.732804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:58.732804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:58.732804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:58.732804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:58.732804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:58.732804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:58.732804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:58.733169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.733169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:58.733518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.733518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:58.733518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:58.733518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:58.733518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:58.733518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:58.733518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.733518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:58.733518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:58.733518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.733518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:58.733518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:58.733518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.733518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:58.733518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:58.733518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:34:58.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:58.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:58.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:58.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:58.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:58.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:58.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:58.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:58.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:58.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:58.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:58.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:58.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:58.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:58.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:58.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:58.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:58.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:58.736488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:58.736488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.736488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:58.926237 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:58.926807 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:58.927089 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:58.928226 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:58.928476 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:58.928849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.929669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:58.929669 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:58.929669 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:58.929669 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:58.929669 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:58.929669 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:58.929669 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.929669 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:58.931161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.931161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:58.931161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:58.931161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:58.931161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.931161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:34:58.931161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:58.931161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:58.931161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:58.931161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:58.931161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:58.931161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:58.931589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:58.931589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:58.931589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:58.931589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:58.931589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:58.931589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:58.931589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:58.931589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:58.931589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:58.931589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:58.931589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:58.931589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:58.931589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:58.932011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.932011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:58.932011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.932011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:58.932011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:58.932011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:58.932011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:58.932011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:58.932011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.932011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:58.932011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:58.932011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.932011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:58.932011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:58.932011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.933264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:58.933264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:58.933264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.933264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:34:58.933264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:58.933264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:58.933264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:58.933264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:58.933264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:58.933264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:58.933264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:58.933264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:58.933264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:58.933264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:58.933264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:58.933264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:58.933264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:58.933264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:58.933264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:58.934136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:58.934136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:58.934136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:58.934136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:58.934136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.934136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:58.972481 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:58.972868 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:58.973035 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:58.973231 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:34:58.973826 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:58.974180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.974910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:58.974910 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:58.974910 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:58.974910 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:58.974910 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:58.974910 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:58.974910 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.974910 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:58.975651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.975651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:34:58.975651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:58.975651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:58.975651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:58.975651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:58.975651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:58.975651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:58.975651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:58.975651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:58.975651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:58.975651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:34:58.975651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:58.975651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:34:58.975651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:34:58.976457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:58.976457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:34:58.976457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:58.976457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:34:58.976457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:34:58.976457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:58.976457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:34:58.976457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.976457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:34:58.976457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.976457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:58.976457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:58.976457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:58.976457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.977166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:34:58.977166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:34:58.977166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:34:58.977166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:58.977166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:58.977166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:58.977166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:58.977166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:58.977166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:58.977166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:58.977166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:58.977166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:58.977166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:58.977849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:58.977849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:58.977849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:58.977849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:58.977849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:58.977849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:58.977849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:58.977849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:58.977849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:58.977849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:58.977849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.977849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:58.978386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.978386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:58.978386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:58.978386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:58.978386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:58.978386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:58.978386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.978386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:58.978386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:58.978386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.978386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:58.978386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:58.978386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.978386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:58.978386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:58.978386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.980061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:34:58.980061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:34:58.980061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:34:58.980061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:58.980061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:58.980061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:58.980061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:58.980061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:58.980061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:58.980061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:58.980061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:58.980061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:58.980061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:58.980061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:58.980061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:58.980061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:58.980061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:58.980061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:58.981168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:58.981168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:58.981168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:58.981168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:58.981168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:58.981168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.981168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:58.981539 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:58.981539 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:58.981706 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:58.981780 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:58.982294 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:58.982710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.983519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:58.983519 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:58.983519 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:58.983519 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:58.983519 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:58.983519 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:58.983519 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.983519 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:58.984261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.984261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:34:58.984261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:58.984261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:58.984261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:58.984261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:58.984261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:58.984261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:58.984261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:58.984261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:34:58.984261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:58.984261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:34:58.984261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:58.984261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:34:58.984261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:34:58.985211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:58.985211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:34:58.985211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:58.985211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:34:58.985211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:34:58.985211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:58.985211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:34:58.985211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.985211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:34:58.985211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.985211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:58.985211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:58.985211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:58.985211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.986090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:34:58.986090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:34:58.986090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:58.986090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:58.986090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:58.986090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:58.986090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:58.986090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:58.986090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:58.986090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:58.986090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:58.986090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:58.986090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:58.986779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:58.986779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:58.986779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:58.986779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:58.986779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:58.986779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:58.986779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:58.986779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:58.986779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:58.986779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.986779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:58.986779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.987123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:58.987123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:58.987123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:58.987123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:58.987123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:58.987123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.987123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:58.987123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:58.987123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.987123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:58.987123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:58.987123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.987123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:58.987123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:58.987123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.987123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:34:58.987123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:34:58.987123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:34:58.987123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:34:58.987575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:58.987575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:58.987575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:58.987575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:58.987575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:58.987575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:58.987575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:58.987575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:58.987575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:58.987575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:58.987575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:58.987575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:58.987575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:58.987575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:58.987575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:58.987575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:58.987575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:58.987575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:58.987575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.987575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:34:58.988317 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:34:58.988503 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:34:58.988614 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:34:58.988789 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:34:58.989241 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:58.989648 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:34:58.990060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.991001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:34:58.991001 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:58.991001 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:34:58.991001 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:34:58.991001 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:58.991001 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:34:58.991001 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.991001 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:34:58.991736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.991736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:34:58.991736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:58.991736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:34:58.991736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.991736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:34:58.991736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:58.991736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:58.991736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:58.991736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:58.991736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:58.991736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:58.992704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:34:58.992704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:58.992704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:34:58.992704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:58.992704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:34:58.992704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:34:58.992704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:58.992704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:34:58.992704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:58.992704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:34:58.992704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:34:58.992704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:58.992704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:34:58.993454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.993454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:34:58.993454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.993454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:34:58.993454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:34:58.993454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:34:58.993454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:34:58.993454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:34:58.993454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.993454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:34:58.993454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:34:58.993454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.993454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:34:58.993454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:34:58.993454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.994060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:34:58.994060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:34:58.994060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:34:58.994060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:34:58.994060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:34:58.994060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:58.994060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:58.994060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:34:58.994060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:34:58.994060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:34:58.994060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:34:58.994060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:34:58.994060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:34:58.994060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:34:58.994060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:34:58.994060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:34:58.994060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:34:58.994060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:34:58.994060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:34:58.994594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:34:58.994594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:34:58.994594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:34:58.994594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:34:58.994594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:34:58.994594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:03.695307 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:03.695932 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:03.696121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:03.697423 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:03.730923 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:03.731536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.732697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:03.732697 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:03.732697 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:03.732697 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:03.732697 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:03.732697 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:03.732697 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.732697 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:03.733762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.733762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:03.733762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:03.733762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:03.733762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.733762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:35:03.733762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:03.733762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:03.733762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:03.733762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:03.733762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:03.733762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:03.734469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:03.734469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:03.734469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:03.734469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:03.734469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:03.734469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:03.734469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:03.734469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:03.734469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:03.734469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:03.734469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:03.734469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:03.734469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:03.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:03.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:03.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:03.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:03.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:03.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:03.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:03.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:03.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:03.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:03.735162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.736006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:03.736006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:03.736006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.736006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:35:03.736006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:03.736006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:03.736006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:03.736006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:03.736006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:03.736006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:03.736006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:03.736006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:03.736006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:03.736006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:03.736006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:03.736006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:03.736006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:03.736006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:03.736006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:03.737042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:03.737042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:03.737042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:03.737042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:03.737042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.737042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:03.926867 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:03.927387 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:03.927579 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:03.929106 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:03.929454 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:03.929648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.930589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:03.930589 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:03.930589 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:03.930589 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:03.930589 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:03.930589 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:03.930589 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.930589 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:03.932362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.932362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:03.932362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:03.932362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:03.932362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.932362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:35:03.932362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:03.932362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:03.932362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:03.932362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:03.932362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:03.932362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:03.933241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:03.933241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:03.933241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:03.933241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:03.933241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:03.933241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:03.933241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:03.933241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:03.933241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:03.933241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:03.933241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:03.933241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:03.933241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:03.933839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.933839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:03.934217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.934217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:03.934217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:03.934217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:03.934217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:03.934217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:03.934217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.934217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:03.934217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:03.934217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.934217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:03.934217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:03.934217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.934217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:03.934217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:03.934217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:35:03.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:03.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:03.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:03.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:03.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:03.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:03.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:03.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:03.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:03.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:03.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:03.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:03.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:03.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:03.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:03.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:03.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:03.935041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:03.935560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:03.935560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.935560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:03.973276 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:03.973897 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:03.974101 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:03.974302 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:35:03.975197 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:03.975423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.976337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:03.976337 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:03.976337 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:03.976337 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:03.976337 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:03.976337 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:03.976337 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.976337 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:03.977922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.977922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:35:03.977922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:03.977922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:03.977922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:03.977922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:03.977922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:03.977922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:03.977922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:03.977922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:03.977922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:03.977922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:35:03.977922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:03.977922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:35:03.977922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:35:03.978699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:03.978699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:35:03.978699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:03.978699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:35:03.978699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:35:03.978699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:03.978699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:35:03.978699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.978699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:35:03.979348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.979348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:03.979348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:03.979348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:03.979348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.979348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:35:03.979348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:35:03.979348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:35:03.979348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:03.979348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:03.979348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:03.979348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:03.979874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:03.979874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:03.979874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:03.979874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:03.979874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:03.979874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:03.979874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:03.979874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:03.979874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:03.979874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:03.979874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:03.979874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:03.979874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:03.979874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:03.980236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:03.980236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:03.980236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.980236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:03.981302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.981302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:03.981302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:03.981302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:03.981302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:03.981302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:03.981302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.981302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:03.981302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:03.981302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.981302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:03.981302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:03.981302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.981302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:03.981302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:03.981302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.981764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:35:03.981764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:35:03.981764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:35:03.981764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:03.981764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:03.981764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:03.981764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:03.981764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:03.981764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:03.981764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:03.981764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:03.981764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:03.981764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:03.981764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:03.981764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:03.981764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:03.981764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:03.981764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:03.982264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:03.982264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:03.982264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:03.982264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:03.982264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:03.982264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.982264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:03.982264 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:03.982264 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:03.982524 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:03.982903 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:03.983532 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:03.983941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.984927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:03.984927 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:03.984927 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:03.984927 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:03.984927 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:03.984927 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:03.984927 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.984927 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:03.986264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.986264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:35:03.986264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:03.986264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:03.986264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:03.986264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:03.986264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:03.986264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:03.986264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:03.986264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:03.986264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:03.986264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:35:03.986264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:03.986264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:35:03.986264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:35:03.986650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:03.986650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:35:03.986650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:03.986650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:35:03.986650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:35:03.986650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:03.986650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:35:03.986650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.986650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:35:03.987809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.987809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:03.987809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:03.987809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:03.987809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.987809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:35:03.987809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:35:03.987809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:03.987809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:03.987809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:03.987809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:03.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:03.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:03.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:03.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:03.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:03.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:03.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:03.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:03.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:03.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:03.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:03.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:03.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:03.988144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:03.988584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:03.988584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:03.988584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.988584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:03.990049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.990049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:03.990049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:03.990049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:03.990049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:03.990049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:03.990049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.990049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:03.990049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:03.990049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.990049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:03.990049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:03.990049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.990049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:03.990049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:03.990049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.990049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:35:03.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:35:03.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:03.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:03.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:03.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:03.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:03.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:03.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:03.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:03.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:03.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:03.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:03.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:03.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:03.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:03.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:03.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:03.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:03.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:03.991095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:03.991095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:03.991095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.991095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:03.992565 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:03.992999 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:03.993115 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:03.993270 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:03.993851 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:03.994211 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:03.994449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.995242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:03.995242 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:03.995242 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:03.995242 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:03.995242 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:03.995242 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:03.995242 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.995242 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:03.996120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.996120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:03.996120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:03.996120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:03.996120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.996120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:35:03.996120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:03.996120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:03.996120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:03.996120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:03.996120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:03.996120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:03.996485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:03.996485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:03.996485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:03.996485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:03.996485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:03.996485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:03.996485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:03.996485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:03.996485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:03.996485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:03.996485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:03.996485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:03.996485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:03.996871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.996871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:03.997119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.997119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:03.997119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:03.997119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:03.997119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:03.997119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:03.997119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.997119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:03.997119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:03.997119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.997119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:03.997119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:03.997119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.997119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:03.997119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:03.997119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:03.997559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:35:03.997559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:03.997559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:03.997559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:03.997559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:03.997559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:03.997559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:03.997559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:03.997559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:03.997559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:03.997559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:03.997559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:03.997559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:03.997559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:03.997559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:03.997559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:03.997559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:03.997559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:03.997559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:03.998097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:03.998097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:03.998097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:08.696122 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:08.697049 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:08.697049 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:08.697821 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:08.730826 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=256 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:35:08.740412 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:08.740936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.742961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:08.742961 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:08.742961 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:08.742961 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:08.742961 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:08.742961 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:08.742961 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.742961 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:08.744112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.744112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:08.744112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:08.744112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:08.744112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.744112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:35:08.744112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:08.744112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:08.744112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:08.744112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:08.744112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:08.744112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:08.744519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:08.744519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:08.744519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:08.744519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:08.744519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:08.744519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:08.744519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:08.744519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:08.744519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:08.744519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:08.744519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:08.744519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:08.744519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:08.744917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.744917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:08.745600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.745600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:08.745600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:08.745600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:08.745600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:08.745600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:08.745600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.745600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:08.745600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:08.745600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.745600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:08.745600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:08.745600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.745600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:08.745600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:08.745600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.746797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:35:08.746797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:08.746797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:08.746797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:08.746797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:08.746797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:08.746797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:08.746797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:08.746797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:08.746797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:08.746797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:08.746797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:08.746797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:08.746797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:08.746797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:08.746797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:08.746797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:08.746797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:08.746797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:08.747433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:08.747433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.747433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:08.927449 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:08.927768 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:08.927878 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:08.929168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:08.929600 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:08.929897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.931043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:08.931043 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:08.931043 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:08.931043 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:08.931043 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:08.931043 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:08.931043 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.931043 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:08.932152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.932152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:08.932152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:08.932152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:08.932152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.932152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:35:08.932152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:08.932152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:08.932152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:08.932152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:08.932152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:08.932152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:08.932860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:08.932860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:08.932860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:08.932860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:08.932860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:08.932860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:08.932860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:08.932860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:08.932860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:08.932860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:08.932860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:08.932860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:08.932860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:08.933418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.933418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:08.933418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.933418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:08.933418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:08.933418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:08.933418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:08.933418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:08.933418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.933418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:08.933418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:08.933418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.933418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:08.933418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:08.933418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.934207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:08.934207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:08.934207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.934207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:35:08.934207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:08.934207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:08.934207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:08.934207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:08.934207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:08.934207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:08.934207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:08.934207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:08.934207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:08.934207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:08.934207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:08.934207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:08.934207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:08.934207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:08.934207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:08.935147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:08.935147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:08.935147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:08.935147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:08.935147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.935147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:08.973896 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:08.974281 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:08.974530 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:08.974818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:35:08.975351 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:08.975628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.976361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:08.976361 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:08.976361 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:08.976361 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:08.976361 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:08.976361 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:08.976361 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.976361 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:08.977588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.977588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:35:08.977588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:08.977588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:08.977588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:08.977588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:08.977588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:08.977588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:08.977588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:08.977588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:08.977588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:08.977588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:35:08.977588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:08.977588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:35:08.977588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:35:08.978481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:08.978481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:35:08.978481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:08.978481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:35:08.978481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:35:08.978481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:08.978481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:35:08.978481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.978481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:35:08.979006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.979006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:08.979006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:08.979006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:08.979006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.979006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:35:08.979006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:35:08.979006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:35:08.979006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:08.979006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:08.979006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:08.979006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:08.979714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:08.979714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:08.979714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:08.979714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:08.979714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:08.979714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:08.979714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:08.979714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:08.979714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:08.979714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:08.979714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:08.979714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:08.979714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:08.979714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:08.980186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:08.980186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:08.980186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.980186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:08.980303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.980303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:08.980303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:08.980303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:08.980303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:08.980303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:08.980303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.980303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:08.980303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:08.980303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.980303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:08.980303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:08.980303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.980303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:08.980303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:08.980303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.980795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:35:08.980795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:35:08.980795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:35:08.980795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:08.980795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:08.980795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:08.980795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:08.980795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:08.980795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:08.980795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:08.980795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:08.980795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:08.980795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:08.980795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:08.980795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:08.980795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:08.980795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:08.980795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:08.981266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:08.981266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:08.981266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:08.981266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:08.981266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:08.981266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.981266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:08.982126 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:08.982361 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:08.982526 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:08.982749 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:08.983361 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:08.983784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.984586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:08.984586 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:08.984586 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:08.984586 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:08.984586 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:08.984586 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:08.984586 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.984586 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:08.985688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.985688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:35:08.985688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:08.985688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:08.985688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:08.985688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:08.985688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:08.985688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:08.985688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:08.985688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:08.985688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:08.985688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:35:08.985688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:08.985688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:35:08.985688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:35:08.986110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:08.986110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:35:08.986110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:08.986110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:35:08.986110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:35:08.986110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:08.986110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:35:08.986110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.986110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:35:08.986976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.986976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:08.986976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:08.986976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:08.986976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.986976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:35:08.986976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:35:08.986976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:08.986976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:08.986976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:08.986976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:08.987339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:08.987339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:08.987339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:08.987339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:08.987339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:08.987339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:08.987339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:08.987339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:08.987339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:08.987339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:08.987339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:08.987339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:08.987339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:08.987339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:08.987722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:08.987722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:08.987722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.987722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:08.988343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.988343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:08.988343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:08.988343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:08.988343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:08.988343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:08.988343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.988343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:08.988343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:08.988343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.988343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:08.988343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:08.988343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.988343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:08.988343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:08.988343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.988343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:35:08.988896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:35:08.988896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:08.988896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:08.988896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:08.988896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:08.988896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:08.988896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:08.988896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:08.988896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:08.988896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:08.988896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:08.988896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:08.988896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:08.988896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:08.988896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:08.988896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:08.988896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:08.988896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:08.988896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:08.989348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:08.989348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:08.989348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.989348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:08.993074 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:08.993326 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:08.993446 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:08.993704 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:08.994227 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:08.994537 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:08.994844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.995695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:08.995695 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:08.995695 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:08.995695 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:08.995695 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:08.995695 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:08.995695 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.995695 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:08.996646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.996646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:08.996646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:08.996646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:08.996646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.996646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:35:08.996646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:08.996646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:08.996646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:08.996646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:08.996646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:08.996646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:08.997104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:08.997104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:08.997104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:08.997104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:08.997104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:08.997104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:08.997104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:08.997104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:08.997104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:08.997104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:08.997104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:08.997104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:08.997104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:08.997455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.997455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:08.997752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.997752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:08.997752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:08.997752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:08.997752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:08.997752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:08.997752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.997752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:08.997752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:08.997752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.997752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:08.997752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:08.997752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.997752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:08.997752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:08.997752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:08.998191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:35:08.998191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:08.998191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:08.998191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:08.998191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:08.998191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:08.998191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:08.998191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:08.998191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:08.998191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:08.998191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:08.998191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:08.998191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:08.998191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:08.998191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:08.998191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:08.998191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:08.998191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:08.998191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:08.998736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:08.998736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:08.998736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:13.696852 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:13.698875 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:13.698875 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:13.699317 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:13.735976 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:13.735976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.736295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:13.736295 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:13.736295 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:13.736295 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:13.736295 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:13.736295 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:13.736295 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.736295 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:13.737134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.737134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:13.737134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:13.737134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:13.737134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.737134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:35:13.737134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:13.737134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:13.737134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:13.737134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:13.737134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:13.737134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:13.737852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:13.737852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:13.737852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:13.737852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:13.737852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:13.737852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:13.737852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:13.737852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:13.737852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:13.737852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:13.737852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:13.737852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:13.737852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:13.738596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.738596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:13.738596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.738596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:13.738596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:13.738596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:13.738596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:13.738596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:13.738596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.738596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:13.738596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:13.738596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.738596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:13.738596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:13.738596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.739431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:13.739431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:13.739431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.739431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:35:13.739431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:13.739431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:13.739431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:13.739431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:13.739431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:13.739431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:13.739431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:13.739431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:13.739431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:13.739431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:13.739431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:13.739431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:13.739431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:13.739431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:13.739431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:13.740411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:13.740411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:13.740411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:13.740411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:13.740411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.740411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:13.928164 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:13.929567 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:13.929567 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:13.933111 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:13.933111 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:13.933111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.933560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:13.933560 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:13.933560 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:13.933560 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:13.933560 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:13.933560 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:13.933560 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.933560 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:13.935202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.935202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:13.935202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:13.935202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:13.935202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.935202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:35:13.935202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:13.935202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:13.935202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:13.935202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:13.935202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:13.935202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:13.935711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:13.935711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:13.935711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:13.935711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:13.935711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:13.935711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:13.935711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:13.935711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:13.935711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:13.935711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:13.935711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:13.935711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:13.935711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:13.936096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.936096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:13.936845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.936845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:13.936845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:13.936845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:13.936845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:13.936845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:13.936845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.936845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:13.936845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:13.936845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.936845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:13.936845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:13.936845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.936845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:13.936845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:13.936845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.937326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:35:13.937326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:13.937326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:13.937326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:13.937326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:13.937326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:13.937326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:13.937326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:13.937326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:13.937326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:13.937326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:13.937326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:13.937326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:13.937326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:13.937326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:13.937326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:13.937326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:13.937326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:13.937326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:13.937903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:13.937903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.937903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:13.974604 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:13.974993 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:13.975180 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:13.975386 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:35:13.976400 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:13.976808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.977821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:13.977821 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:13.977821 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:13.977821 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:13.977821 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:13.977821 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:13.977821 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.977821 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:13.979354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.979354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:35:13.979354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:13.979354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:13.979354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:13.979354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:13.979354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:13.979354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:13.979354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:13.979354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:13.979354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:13.979354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:35:13.979354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:13.979354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:35:13.979354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:35:13.980313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:13.980313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:35:13.980313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:13.980313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:35:13.980313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:35:13.980313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:13.980313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:35:13.980313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.980313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:35:13.981110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.981110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:13.981110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:13.981110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:13.981110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.981110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:35:13.981110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:35:13.981110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:35:13.981110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:13.981110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:13.981110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:13.981110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:13.981985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:13.981985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:13.981985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:13.981985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:13.981985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:13.981985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:13.981985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:13.981985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:13.981985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:13.981985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:13.981985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:13.981985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:13.981985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:13.981985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:13.982399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:13.982399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:13.982399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.982399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:13.983158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.983158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:13.983158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:13.983158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:13.983158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:13.983158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:13.983158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.983158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:13.983158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:13.983158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.983158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:13.983158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:13.983158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.983158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:13.983158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:13.983158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.983908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:35:13.983908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:35:13.983908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:35:13.983908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:13.983908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:13.983908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:13.983908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:13.983908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:13.983908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:13.983908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:13.983908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:13.983908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:13.983908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:13.983908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:13.983908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:13.983908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:13.983908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:13.983908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:13.984408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:13.984408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:13.984408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:13.984408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:13.984408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:13.984408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.984408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:13.984408 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:13.984408 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:13.984408 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:13.984733 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:13.984733 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:13.984999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.985400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:13.985400 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:13.985400 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:13.985400 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:13.985400 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:13.985400 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:13.985400 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.985400 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:13.986445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.986445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:35:13.986445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:13.986445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:13.986445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:13.986445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:13.986445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:13.986445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:13.986445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:13.986445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:13.986445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:13.986445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:35:13.986445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:13.986445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:35:13.986445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:35:13.986977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:13.986977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:35:13.986977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:13.986977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:35:13.986977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:35:13.986977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:13.986977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:35:13.986977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.986977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:35:13.987350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.987350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:13.987350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:13.987350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:13.987350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.987350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:35:13.987350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:35:13.987350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:13.987350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:13.987350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:13.987350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:13.987776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:13.987776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:13.987776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:13.987776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:13.987776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:13.987776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:13.987776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:13.987776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:13.987776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:13.987776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:13.987776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:13.987776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:13.987776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:13.987776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:13.988315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:13.988315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:13.988315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.988315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:13.988511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.988511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:13.988511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:13.988511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:13.988511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:13.988511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:13.988511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.988511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:13.988511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:13.988511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.988511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:13.988511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:13.988511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.988511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:13.988511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:13.988511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.988511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:35:13.989143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:35:13.989143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:13.989143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:13.989143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:13.989143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:13.989143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:13.989143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:13.989143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:13.989143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:13.989143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:13.989143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:13.989143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:13.989143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:13.989143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:13.989143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:13.989143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:13.989143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:13.989143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:13.989143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:13.989854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:13.989854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:13.989854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.989854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:13.993371 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:13.993697 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:13.993834 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:13.994284 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:13.994509 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:13.994795 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:13.994953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.995563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:13.995563 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:13.995563 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:13.995563 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:13.995563 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:13.995563 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:13.995563 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.995563 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:13.996538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.996538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:13.996538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:13.996538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:13.996538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.996538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:35:13.996538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:13.996538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:13.996538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:13.996538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:13.996538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:13.996538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:13.998008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:13.998008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:13.998008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:13.998008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:13.998008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:13.998008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:13.998008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:13.998008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:13.998008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:13.998008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:13.998008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:13.998008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:13.998008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:13.998448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.998448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:13.998448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:13.998448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:13.998448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:13.998448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:13.998448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:13.998448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:13.998448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.998448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:13.998448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:13.998448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.998448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:13.998448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:13.998448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.999468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:13.999468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:13.999468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:13.999468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:35:13.999468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:13.999468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:13.999468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:13.999468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:13.999468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:13.999468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:13.999468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:13.999468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:13.999468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:13.999468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:13.999468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:13.999468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:13.999468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:13.999468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:13.999468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:14.000010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:14.000010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:14.000010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:14.000010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:14.000010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:14.000010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:18.698268 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:18.698940 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:18.699121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:18.700555 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:18.743487 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:18.743772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.744592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:18.744592 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:18.744592 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:18.744592 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:18.744592 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:18.744592 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:18.744592 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.744592 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:18.745522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.745522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:18.745522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:18.745522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:18.745522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.745522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:35:18.745522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:18.745522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:18.745522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:18.745522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:18.745522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:18.745522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:18.746275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:18.746275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:18.746275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:18.746275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:18.746275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:18.746275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:18.746275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:18.746275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:18.746275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:18.746275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:18.746275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:18.746275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:18.746275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:18.746964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.746964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:18.746964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.746964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:18.746964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:18.746964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:18.746964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:18.746964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:18.746964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.746964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:18.746964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:18.746964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.746964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:18.746964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:18.746964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.747742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:18.747742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:18.747742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.747742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:35:18.747742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:18.747742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:18.747742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:18.747742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:18.747742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:18.747742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:18.747742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:18.747742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:18.747742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:18.747742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:18.747742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:18.747742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:18.747742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:18.747742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:18.747742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:18.748741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:18.748741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:18.748741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:18.748741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:18.748741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.748741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:18.928922 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:18.929569 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:18.929832 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:18.930942 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:18.931303 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:18.931564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.932614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:18.932614 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:18.932614 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:18.932614 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:18.932614 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:18.932614 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:18.932614 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.932614 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:18.934172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.934172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:18.934172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:18.934172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:18.934172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.934172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:35:18.934172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:18.934172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:18.934172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:18.934172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:18.934172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:18.934172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:18.934748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:18.934748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:18.934748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:18.934748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:18.934748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:18.934748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:18.934748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:18.934748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:18.934748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:18.934748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:18.934748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:18.934748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:18.934748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:18.935351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.935351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:18.935918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.935918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:18.935918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:18.935918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:18.935918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:18.935918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:18.935918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.935918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:18.935918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:18.935918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.935918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:18.935918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:18.935918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.935918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:18.935918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:18.935918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.936641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:35:18.936641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:18.936641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:18.936641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:18.936641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:18.936641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:18.936641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:18.936641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:18.936641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:18.936641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:18.936641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:18.936641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:18.936641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:18.936641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:18.936641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:18.936641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:18.936641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:18.936641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:18.936641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:18.937448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:18.937448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.937448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:18.975050 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:18.975494 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:18.975710 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:18.975909 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:35:18.976704 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:18.977119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.977923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:18.977923 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:18.977923 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:18.977923 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:18.977923 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:18.977923 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:18.977923 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.977923 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:18.978716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.978716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:35:18.978716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:18.978716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:18.978716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:18.978716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:18.978716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:18.978716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:18.978716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:18.978716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:18.978716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:18.978716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:35:18.978716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:18.978716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:35:18.978716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:35:18.979989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:18.979989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:35:18.979989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:18.979989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:35:18.979989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:35:18.979989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:18.979989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:35:18.979989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.979989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:35:18.979989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.979989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:18.979989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:18.979989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:18.979989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.984850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:35:18.984850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:35:18.984850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:35:18.984850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:18.984850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:18.984850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:18.984850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:18.984850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:18.984850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:18.984850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:18.984850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:18.984850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:18.984850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:18.985722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:18.985722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:18.985722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:18.985722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:18.985722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:18.985722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:18.985722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:18.985722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:18.985722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:18.985722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:18.985722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.985722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:18.986300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.986300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:18.986300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:18.986300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:18.986300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:18.986300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:18.986300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.986300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:18.986300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:18.986300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.986300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:18.986300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:18.986300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.986300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:18.986300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:18.986300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.987223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:35:18.987223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:35:18.987223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:35:18.987223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:18.987223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:18.987223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:18.987223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:18.987223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:18.987223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:18.987223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:18.987223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:18.987223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:18.987223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:18.987223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:18.987223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:18.987223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:18.987223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:18.987223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:18.987959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:18.987959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:18.987959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:18.987959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:18.987959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:18.987959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.987959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:18.987959 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:18.987959 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:18.987959 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:18.988478 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:18.988478 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:18.988478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.988478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:18.988478 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:18.988478 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:18.988478 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:18.988478 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:18.988478 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:18.988478 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.988478 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:18.988478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.988983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:35:18.988983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:18.988983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:18.988983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:18.988983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:18.988983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:18.988983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:18.988983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:18.988983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:18.988983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:18.988983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:35:18.988983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:18.988983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:35:18.988983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:35:18.988983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:18.988983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:35:18.988983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:18.989736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:35:18.989736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:35:18.989736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:18.989736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:35:18.989736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.989736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:35:18.989736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.989736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:18.989736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:18.989736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:18.989736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.989736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:35:18.989736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:35:18.990154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:18.990154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:18.990154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:18.990154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:18.990154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:18.990154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:18.990154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:18.990154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:18.990154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:18.990154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:18.990154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:18.990154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:18.990154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:18.990556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:18.990556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:18.990556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:18.990556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:18.990556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:18.990556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:18.990556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:18.990556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.990556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:18.990556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.990556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:18.990556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:18.990915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:18.990915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:18.990915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:18.990915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.990915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:18.990915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:18.990915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.990915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:18.990915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:18.990915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.990915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:18.990915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:18.990915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.990915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:35:18.990915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:35:18.990915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:18.990915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:18.990915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:18.990915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:18.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:18.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:18.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:18.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:18.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:18.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:18.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:18.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:18.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:18.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:18.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:18.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:18.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:18.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:18.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:18.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:18.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.991446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:18.993708 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:18.993960 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:18.994072 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:18.994219 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:18.994584 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:18.994845 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:18.995013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.995921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:18.995921 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:18.995921 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:18.995921 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:18.995921 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:18.995921 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:18.995921 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.995921 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:18.996841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.996841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:18.996841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:18.996841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:18.996841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.996841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:35:18.996841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:18.996841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:18.996841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:18.996841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:18.996841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:18.996841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:18.997247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:18.997247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:18.997247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:18.997247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:18.997247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:18.997247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:18.997247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:18.997247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:18.997247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:18.997247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:18.997247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:18.997247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:18.997247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:18.997633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.997633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:18.997809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.997809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:18.997809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:18.997809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:18.997809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:18.997809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:18.997809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.997809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:18.997809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:18.997809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.997809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:18.997809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:18.997809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.997809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:18.997809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:18.997809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:18.998377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:35:18.998377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:18.998377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:18.998377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:18.998377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:18.998377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:18.998377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:18.998377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:18.998377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:18.998377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:18.998377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:18.998377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:18.998377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:18.998377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:18.998377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:18.998377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:18.998377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:18.998377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:18.998377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:18.998891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:18.998891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:18.998891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:23.699094 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:23.700032 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:23.700032 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:23.701064 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:23.740201 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:23.740377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.740947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:23.740947 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:23.740947 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:23.740947 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:23.740947 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:23.740947 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:23.740947 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.740947 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:23.742394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.742394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:23.742394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:23.742394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:23.742394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:23.742394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:35:23.742394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:23.742394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:23.742394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:23.742394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:23.742394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:23.742394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:23.742842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:23.742842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:23.742842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:23.742842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:23.742842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:23.742842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:23.742842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:23.742842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:23.742842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:23.742842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:23.742842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:23.742842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:23.742842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:23.743181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.743181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:23.746104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.746104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:23.746104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:23.746104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:23.746104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:23.746104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:23.746104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:23.746104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:23.746104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:23.746104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:23.746104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:23.746104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:23.746104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:23.746104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:23.746104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:23.746104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:23.746560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:35:23.746560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:23.746560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:23.746560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:23.746560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:23.746560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:23.746560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:23.746560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:23.746560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:23.746560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:23.746560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:23.746560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:23.746560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:23.746560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:23.746560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:23.746560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:23.746560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:23.746560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:23.746560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:23.747087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:23.747087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.747087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:23.929643 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:23.930219 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:23.930467 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:23.931501 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:23.932133 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:23.932133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.933241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:23.933241 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:23.933241 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:23.933241 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:23.933241 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:23.933241 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:23.933241 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.933241 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:23.934311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.934311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:23.934311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:23.934311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:23.934311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:23.934311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:35:23.934311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:23.934311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:23.934311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:23.934311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:23.934311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:23.934311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:23.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:23.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:23.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:23.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:23.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:23.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:23.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:23.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:23.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:23.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:23.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:23.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:23.934940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:23.935607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.935607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:23.936141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.936141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:23.936141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:23.936141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:23.936141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:23.936141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:23.936141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:23.936141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:23.936141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:23.936141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:23.936141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:23.936141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:23.936141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:23.936141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:23.936141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:23.936141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:23.936809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:35:23.936809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:23.936809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:23.936809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:23.936809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:23.936809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:23.936809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:23.936809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:23.936809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:23.936809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:23.936809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:23.936809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:23.936809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:23.936809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:23.936809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:23.936809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:23.936809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:23.936809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:23.936809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:23.937424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:23.937424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.937424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:23.975567 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:23.976007 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:23.976159 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:23.976329 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:35:23.976988 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:23.977333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.978159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:23.978159 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:23.978159 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:23.978159 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:23.978159 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:23.978159 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:23.978159 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.978159 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:23.979608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.979608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:35:23.979608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:23.979608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:23.979608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:23.979608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:23.979608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:23.979608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:23.979608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:23.979608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:23.979608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:23.979608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:35:23.979608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:23.979608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:35:23.979608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:35:23.980634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:23.980634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:35:23.980634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:23.980634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:35:23.980634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:35:23.980634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:23.980634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:35:23.980634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.980634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:35:23.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:23.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:23.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:23.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:23.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:35:23.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:35:23.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:35:23.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:23.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:23.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:23.981229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:23.981818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:23.981818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:23.981818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:23.981818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:23.981818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:23.981818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:23.981818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:23.981818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:23.981818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:23.981818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:23.981818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:23.981818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:23.981818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:23.981818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:23.982351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:23.982351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:23.982351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.982351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:23.982743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.982743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:23.982743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:23.982743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:23.982743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:23.982743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:23.982743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:23.982743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:23.982743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:23.982743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:23.982743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:23.982743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:23.982743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:23.982743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:23.982743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:23.982743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:23.983378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:35:23.983378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:35:23.983378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:35:23.983378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:23.983378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:23.983378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:23.983378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:23.983378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:23.983378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:23.983378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:23.983378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:23.983378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:23.983378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:23.983378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:23.983378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:23.983378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:23.983378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:23.983378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:23.983874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:23.983874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:23.983874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:23.983874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:23.983874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:23.983874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.983874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:23.984828 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:23.985135 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:23.985282 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:23.985454 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:23.985916 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:23.986264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.986944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:23.986944 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:23.986944 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:23.986944 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:23.986944 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:23.986944 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:23.986944 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.986944 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:23.987828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.987828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:35:23.987828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:23.987828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:23.987828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:23.987828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:23.987828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:23.987828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:23.987828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:23.987828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:23.987828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:23.987828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:35:23.987828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:23.987828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:35:23.987828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:35:23.988397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:23.988397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:35:23.988397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:23.988397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:35:23.988397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:35:23.988397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:23.988397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:35:23.988397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.988397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:35:23.989019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.989019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:23.989019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:23.989019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:23.989019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:23.989019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:35:23.989019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:35:23.989019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:23.989019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:23.989019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:23.989019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:23.989508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:23.989508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:23.989508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:23.989508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:23.989508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:23.989508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:23.989508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:23.989508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:23.989508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:23.989508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:23.989508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:23.989508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:23.989508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:23.989508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:23.990094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:23.990094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:23.990094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.990094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:23.990496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.990496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:23.990496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:23.990496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:23.990496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:23.990496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:23.990496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:23.990496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:23.990496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:23.990496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:23.990496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:23.990496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:23.990496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:23.990496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:23.990496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:23.990496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:23.990496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:35:23.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:35:23.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:23.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:23.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:23.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:23.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:23.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:23.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:23.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:23.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:23.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:23.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:23.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:23.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:23.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:23.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:23.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:23.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:23.991048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:23.991545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:23.991545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:23.991545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.991545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:23.994149 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:23.994451 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:23.994597 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:23.994861 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:23.995226 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:23.995547 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:23.995791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.996429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:23.996429 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:23.996429 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:23.996429 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:23.996429 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:23.996429 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:23.996429 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.996429 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:23.997910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.997910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:23.997910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:23.997910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:23.997910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:23.997910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:35:23.997910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:23.997910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:23.997910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:23.997910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:23.997910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:23.997910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:23.998437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:23.998437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:23.998437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:23.998437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:23.998437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:23.998437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:23.998437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:23.998437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:23.998437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:23.998437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:23.998437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:23.998437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:23.998437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:23.998988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:23.998988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:24.000359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:24.000359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:24.000359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:24.000359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:24.000359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:24.000359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:24.000359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:24.000359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:24.000359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:24.000359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:24.000359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:24.000359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:24.000359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:24.000359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:24.000359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:24.000359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:24.001023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:35:24.001023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:24.001023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:24.001023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:24.001023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:24.001023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:24.001023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:24.001023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:24.001023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:24.001023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:24.001023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:24.001023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:24.001023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:24.001023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:24.001023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:24.001023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:24.001023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:24.001023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:24.001023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:24.001708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:24.001708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:24.001708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:28.699697 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:28.700202 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:28.700352 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:28.701125 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:28.734288 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:28.734697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.735540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:28.735540 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:28.735540 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:28.735540 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:28.735540 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:28.735540 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:28.735540 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.735540 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:28.736327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.736327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:28.736327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:28.736327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:28.736327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.736327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:35:28.736327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:28.736327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:28.736327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:28.736327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:28.736327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:28.736327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:28.736880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:28.736880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:28.736880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:28.736880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:28.736880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:28.736880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:28.736880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:28.736880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:28.736880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:28.736880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:28.736880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:28.736880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:28.736880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:28.737396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.737396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:28.737396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.737396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:28.737396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:28.737396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:28.737396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:28.737396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:28.737396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.737396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:28.737396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:28.737396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.737396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:28.737396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:28.737396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.738064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:28.738064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:28.738064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.738064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:35:28.738064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:28.738064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:28.738064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:28.738064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:28.738064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:28.738064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:28.738064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:28.738064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:28.738064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:28.738064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:28.738064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:28.738064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:28.738064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:28.738064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:28.738064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:28.738849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:28.738849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:28.738849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:28.738849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:28.738849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.738849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:28.930637 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:28.931200 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:28.931404 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:28.932577 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:28.932986 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:28.933236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.934150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:28.934150 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:28.934150 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:28.934150 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:28.934150 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:28.934150 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:28.934150 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.934150 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:28.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:28.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:28.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:28.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:35:28.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:28.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:28.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:28.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:28.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:28.935710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:28.936233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:28.936233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:28.936233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:28.936233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:28.936233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:28.936233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:28.936233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:28.936233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:28.936233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:28.936233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:28.936233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:28.936233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:28.936233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:28.937035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.937035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:28.937375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.937375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:28.937375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:28.937375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:28.937375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:28.937375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:28.937375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.937375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:28.937375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:28.937375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.937375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:28.937375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:28.937375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.937375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:28.937375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:28.937375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.938206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:35:28.938206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:28.938206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:28.938206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:28.938206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:28.938206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:28.938206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:28.938206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:28.938206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:28.938206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:28.938206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:28.938206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:28.938206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:28.938206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:28.938206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:28.938206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:28.938206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:28.938206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:28.938206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:28.939213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:28.939213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.939213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:28.976790 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:28.976790 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:28.976790 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:28.977168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:35:28.977575 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:28.978068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.979143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:28.979143 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:28.979143 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:28.979143 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:28.979143 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:28.979143 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:28.979143 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.979143 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:28.980582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.980582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:35:28.980582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:28.980582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:28.980582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:28.980582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:28.980582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:28.980582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:28.980582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:28.980582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:28.980582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:28.980582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:35:28.980582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:28.980582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:35:28.980582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:35:28.981360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:28.981360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:35:28.981360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:28.981360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:35:28.981360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:35:28.981360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:28.981360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:35:28.981360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.981360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:35:28.982452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.982452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:28.982452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:28.982452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:28.982452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.982452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:35:28.982452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:35:28.982452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:35:28.982452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:28.982452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:28.982452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:28.982452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:28.983157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:28.983157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:28.983157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:28.983157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:28.983157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:28.983157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:28.983157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:28.983157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:28.983157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:28.983157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:28.983157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:28.983157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:28.983157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:28.983157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:28.983824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:28.983824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:28.983824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.983824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:28.984637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.984637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:28.984637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:28.984637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:28.984637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:28.984637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:28.984637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.984637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:28.984637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:28.984637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.984637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:28.984637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:28.984637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.984637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:28.984637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:28.984637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:35:28.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:35:28.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:35:28.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:28.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:28.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:28.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:28.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:28.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:28.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:28.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:28.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:28.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:28.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:28.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:28.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:28.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:28.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:28.986484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:28.986484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:28.986484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:28.986484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:28.986484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:28.986484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.986484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:28.986484 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:28.986484 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:28.986484 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:28.986883 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:28.986883 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:28.987126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.987786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:28.987786 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:28.987786 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:28.987786 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:28.987786 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:28.987786 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:28.987786 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.987786 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:28.988338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.988338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:35:28.988338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:28.988338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:28.988338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:28.988338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:28.988338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:28.988338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:28.988338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:28.988338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:28.988338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:28.988338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:35:28.988338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:28.988338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:35:28.988338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:35:28.988735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:28.988735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:35:28.988735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:28.988735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:35:28.988735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:35:28.988735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:28.988735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:35:28.988735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.988735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:35:28.988989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.988989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:28.988989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:28.988989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:28.988989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.988989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:35:28.988989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:35:28.988989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:28.988989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:28.988989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:28.988989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:28.989299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:28.989299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:28.989299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:28.989299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:28.989299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:28.989299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:28.989299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:28.989299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:28.989299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:28.989299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:28.989299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:28.989299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:28.989299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:28.989299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:28.989647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:28.989647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:28.989647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.989647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:28.989807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.989807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:28.989807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:28.989807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:28.989807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:28.989807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:28.989807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.989807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:28.989807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:28.989807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.989807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:28.989807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:28.989807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.989807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:28.989807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:28.989807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.989807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:35:28.990239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:35:28.990239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:28.990239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:28.990239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:28.990239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:28.990239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:28.990239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:28.990239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:28.990239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:28.990239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:28.990239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:28.990239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:28.990239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:28.990239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:28.990239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:28.990239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:28.990239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:28.990239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:28.990239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:28.990792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:28.990792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:28.990792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.990792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:28.994637 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:28.994926 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:28.995022 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:28.995143 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:28.995554 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:28.995899 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:28.996111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.996838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:28.996838 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:28.996838 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:28.996838 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:28.996838 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:28.996838 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:28.996838 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.996838 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:28.997797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.997797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:28.997797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:28.997797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:28.997797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.997797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:35:28.997797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:28.997797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:28.997797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:28.997797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:28.997797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:28.997797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:28.998130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:28.998130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:28.998130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:28.998130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:28.998130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:28.998130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:28.998130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:28.998130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:28.998130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:28.998130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:28.998130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:28.998130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:28.998130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:28.998447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.998447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:28.998761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.998761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:28.998761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:28.998761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:28.998761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:28.998761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:28.998761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.998761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:28.998761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:28.998761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.998761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:28.998761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:28.998761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.998761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:28.998761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:28.998761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:28.999175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:35:28.999175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:28.999175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:28.999175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:28.999175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:28.999175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:28.999175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:28.999175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:28.999175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:28.999175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:28.999175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:28.999175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:28.999175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:28.999175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:28.999175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:28.999175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:28.999175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:28.999175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:28.999175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:28.999740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:28.999740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:28.999740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:33.700725 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:33.702114 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:33.702114 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:33.702921 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:33.734974 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:33.735379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.736278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:33.736278 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:33.736278 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:33.736278 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:33.736278 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:33.736278 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:33.736278 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.736278 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:33.737333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.737333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:33.737333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:33.737333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:33.737333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:33.737333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:35:33.737333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:33.737333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:33.737333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:33.737333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:33.737333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:33.737333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:33.737955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:33.737955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:33.737955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:33.737955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:33.737955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:33.737955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:33.737955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:33.737955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:33.737955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:33.737955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:33.737955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:33.737955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:33.737955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:33.738467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.738467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:33.738890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.738890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:33.738890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:33.738890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:33.738890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:33.738890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:33.738890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:33.738890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:33.738890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:33.738890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:33.738890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:33.738890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:33.738890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:33.738890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:33.738890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:33.738890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:33.739368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:35:33.739368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:33.739368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:33.739368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:33.739368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:33.739368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:33.739368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:33.739368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:33.739368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:33.739368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:33.739368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:33.739368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:33.739368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:33.739368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:33.739368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:33.739368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:33.739368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:33.739368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:33.739368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:33.739910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:33.739910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.739910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:33.931415 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:33.931913 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:33.932124 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:33.933328 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:33.933777 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:33.934063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.934999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:33.934999 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:33.934999 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:33.934999 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:33.934999 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:33.934999 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:33.934999 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.934999 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:33.936504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.936504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:33.936504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:33.936504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:33.936504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:33.936504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:35:33.936504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:33.936504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:33.936504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:33.936504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:33.936504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:33.936504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:33.937174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:33.937174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:33.937174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:33.937174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:33.937174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:33.937174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:33.937174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:33.937174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:33.937174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:33.937174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:33.937174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:33.937174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:33.937174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:33.937763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.937763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:33.938179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.938179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:33.938179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:33.938179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:33.938179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:33.938179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:33.938179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:33.938179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:33.938179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:33.938179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:33.938179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:33.938179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:33.938179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:33.938179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:33.938179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:33.938179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:33.938917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:35:33.938917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:33.938917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:33.938917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:33.938917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:33.938917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:33.938917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:33.938917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:33.938917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:33.938917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:33.938917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:33.938917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:33.938917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:33.938917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:33.938917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:33.938917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:33.938917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:33.938917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:33.938917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:33.939766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:33.939766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.939766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:33.976544 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:33.976964 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:33.977143 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:33.977321 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:35:33.977953 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:33.978273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.979137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:33.979137 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:33.979137 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:33.979137 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:33.979137 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:33.979137 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:33.979137 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.979137 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:33.980535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.980535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:35:33.980535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:33.980535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:33.980535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:33.980535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:33.980535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:33.980535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:33.980535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:33.980535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:33.980535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:33.980535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:35:33.980535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:33.980535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:35:33.980535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:35:33.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:33.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:35:33.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:33.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:35:33.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:35:33.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:33.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:35:33.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.981226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:35:33.981876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.981876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:33.981876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:33.981876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:33.981876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:33.981876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:35:33.981876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:35:33.981876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:35:33.981876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:33.981876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:33.981876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:33.981876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:33.982422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:33.982422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:33.982422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:33.982422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:33.982422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:33.982422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:33.982422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:33.982422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:33.982422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:33.982422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:33.982422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:33.982422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:33.982422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:33.982422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:33.983027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:33.983027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:33.983027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.983027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:33.983698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.983698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:33.983698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:33.983698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:33.983698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:33.983698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:33.983698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:33.983698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:33.983698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:33.983698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:33.983698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:33.983698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:33.983698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:33.983698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:33.983698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:33.983698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:33.984415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:35:33.984415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:35:33.984415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:35:33.984415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:33.984415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:33.984415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:33.984415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:33.984415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:33.984415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:33.984415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:33.984415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:33.984415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:33.984415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:33.984415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:33.984415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:33.984415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:33.984415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:33.984415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:33.985074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:33.985074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:33.985074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:33.985074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:33.985074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:33.985074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.985074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:33.985580 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:33.985994 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:33.986184 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:33.986387 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:33.986923 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:33.987271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.988020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:33.988020 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:33.988020 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:33.988020 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:33.988020 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:33.988020 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:33.988020 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.988020 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:33.988751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.988751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:35:33.988751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:33.988751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:33.988751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:33.988751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:33.988751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:33.988751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:33.988751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:33.988751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:33.988751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:33.988751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:35:33.988751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:33.988751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:35:33.988751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:35:33.989191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:33.989191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:35:33.989191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:33.989191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:35:33.989191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:35:33.989191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:33.989191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:35:33.989191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.989191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:35:33.989580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.989580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:33.989580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:33.989580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:33.989580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:33.989580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:35:33.989580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:35:33.989580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:33.989580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:33.989580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:33.989580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:33.989914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:33.989914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:33.989914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:33.989914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:33.989914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:33.989914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:33.989914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:33.989914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:33.989914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:33.989914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:33.989914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:33.989914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:33.989914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:33.989914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:33.990324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:33.990324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:33.990324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.990324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:33.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:33.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:33.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:33.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:33.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:33.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:33.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:33.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:33.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:33.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:33.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:33.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:33.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:33.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:33.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:33.990621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:35:33.991103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:35:33.991103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:33.991103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:33.991103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:33.991103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:33.991103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:33.991103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:33.991103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:33.991103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:33.991103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:33.991103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:33.991103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:33.991103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:33.991103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:33.991103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:33.991103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:33.991103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:33.991103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:33.991103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:33.991617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:33.991617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:33.991617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.991617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:33.995102 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:33.995395 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:33.995503 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:33.995668 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:33.996150 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:33.996482 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:33.996755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.997438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:33.997438 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:33.997438 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:33.997438 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:33.997438 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:33.997438 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:33.997438 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.997438 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:33.998277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.998277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:33.998277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:33.998277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:33.998277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:33.998277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:35:33.998277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:33.998277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:33.998277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:33.998277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:33.998277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:33.998277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:33.998901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:33.998901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:33.998901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:33.998901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:33.998901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:33.998901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:33.998901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:33.998901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:33.998901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:33.998901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:33.998901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:33.998901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:33.998901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:33.999499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.999499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:33.999499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:33.999499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:33.999499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:33.999499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:33.999499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:33.999499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:33.999499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:33.999499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:33.999499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:33.999499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:33.999499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:33.999499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:33.999499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:34.000278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:34.000278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:34.000278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:34.000278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:35:34.000278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:34.000278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:34.000278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:34.000278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:34.000278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:34.000278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:34.000278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:34.000278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:34.000278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:34.000278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:34.000278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:34.000278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:34.000278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:34.000278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:34.000278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:34.001189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:34.001189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:34.001189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:34.001189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:34.001189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:34.001189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:38.701582 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:38.702322 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:38.702322 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:38.702467 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:38.737882 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:38.738310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.739110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:38.739110 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:38.739110 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:38.739110 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:38.739110 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:38.739110 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:38.739110 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.739110 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:38.739785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.739785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:38.739785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:38.739785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:38.739785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.739785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:35:38.739785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:38.739785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:38.739785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:38.739785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:38.739785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:38.739785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:38.741431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:38.741431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:38.741431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:38.741431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:38.741431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:38.741431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:38.741431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:38.741431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:38.741431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:38.741431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:38.741431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:38.741431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:38.741431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:38.742151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.742151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:38.742151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.742151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:38.742151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:38.742151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:38.742151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:38.742151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:38.742151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.742151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:38.742151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:38.742151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.742151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:38.742151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:38.742151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.744170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:38.744170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:38.744170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.744170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:35:38.744170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:38.744170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:38.744170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:38.744170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:38.744170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:38.744170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:38.744170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:38.744170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:38.744170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:38.744170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:38.744170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:38.744170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:38.744170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:38.744170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:38.744170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:38.746048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:38.746048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:38.746048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:38.746048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:38.746048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.746048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:38.932298 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:38.932967 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:38.932967 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:38.934505 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:38.934505 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:38.934505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.935078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:38.935078 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:38.935078 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:38.935078 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:38.935078 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:38.935078 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:38.935078 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.935078 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:38.936186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.936186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:38.936186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:38.936186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:38.936186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.936186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:35:38.936186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:38.936186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:38.936186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:38.936186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:38.936186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:38.936186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:38.936795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:38.936795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:38.936795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:38.936795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:38.936795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:38.936795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:38.936795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:38.936795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:38.936795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:38.936795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:38.936795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:38.936795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:38.936795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:38.937383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.937383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:38.937383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.937383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:38.937383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:38.937383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:38.937383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:38.937383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:38.937383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.937383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:38.937383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:38.937383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.937383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:38.937383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:38.937383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.938045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:38.938045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:38.938045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.938045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:35:38.938045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:38.938045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:38.938045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:38.938045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:38.938045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:38.938045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:38.938045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:38.938045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:38.938045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:38.938045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:38.938045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:38.938045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:38.938045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:38.938045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:38.938045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:38.938511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:38.938511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:38.938511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:38.938511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:38.938511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.938511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:38.977039 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:38.977600 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:38.977778 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:38.977918 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:35:38.978384 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:38.978612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.979189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:38.979189 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:38.979189 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:38.979189 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:38.979189 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:38.979189 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:38.979189 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.979189 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:38.980251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.980251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:35:38.980251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:38.980251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:38.980251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:38.980251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:38.980251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:38.980251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:38.980251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:38.980251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:38.980251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:38.980251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:35:38.980251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:38.980251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:35:38.980251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:35:38.981230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:38.981230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:35:38.981230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:38.981230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:35:38.981230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:35:38.981230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:38.981230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:35:38.981230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.981230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:35:38.981667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.981667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:38.981667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:38.981667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:38.981667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.981667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:35:38.981667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:35:38.981667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:35:38.981667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:38.981667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:38.981667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:38.981667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:38.982247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:38.982247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:38.982247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:38.982247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:38.982247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:38.982247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:38.982247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:38.982247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:38.982247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:38.982247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:38.982247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:38.982247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:38.982247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:38.982247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:38.982870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:38.982870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:38.982870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.982870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:38.983042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.983042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:38.983042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:38.983042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:38.983042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:38.983042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:38.983042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.983042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:38.983042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:38.983042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.983042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:38.983042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:38.983042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.983042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:38.983042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:38.983042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.983738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:35:38.983738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:35:38.983738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:35:38.983738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:38.983738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:38.983738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:38.983738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:38.983738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:38.983738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:38.983738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:38.983738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:38.983738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:38.983738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:38.983738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:38.983738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:38.983738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:38.983738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:38.983738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:38.984456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:38.984456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:38.984456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:38.984456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:38.984456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:38.984456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.984456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:38.986011 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:38.986317 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:38.986401 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:38.986555 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:38.986970 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:38.987355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.988563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:38.988563 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:38.988563 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:38.988563 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:38.988563 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:38.988563 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:38.988563 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.988563 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:38.989350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.989350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:35:38.989350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:38.989350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:38.989350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:38.989350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:38.989350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:38.989350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:38.989350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:38.989350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:38.989350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:38.989350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:35:38.989350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:38.989350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:35:38.989350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:35:38.989761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:38.989761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:35:38.989761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:38.989761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:35:38.989761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:35:38.989761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:38.989761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:35:38.989761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.989761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:35:38.990297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.990297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:38.990297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:38.990297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:38.990297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.990297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:35:38.990297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:35:38.990297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:38.990297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:38.990297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:38.990297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:38.990605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:38.990605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:38.990605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:38.990605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:38.990605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:38.990605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:38.990605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:38.990605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:38.990605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:38.990605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:38.990605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:38.990605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:38.990605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:38.990605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:38.990990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:38.990990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:38.990990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.990990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:38.991447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.991447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:38.991447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:38.991447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:38.991447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:38.991447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:38.991447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.991447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:38.991447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:38.991447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.991447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:38.991447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:38.991447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.991447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:38.991447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:38.991447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.991447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:35:38.991936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:35:38.991936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:38.991936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:38.991936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:38.991936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:38.991936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:38.991936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:38.991936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:38.991936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:38.991936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:38.991936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:38.991936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:38.991936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:38.991936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:38.991936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:38.991936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:38.991936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:38.991936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:38.991936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:38.992437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:38.992437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:38.992437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.992437 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:38.995579 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:38.995896 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:38.996015 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:38.996172 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:38.996454 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:38.996704 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:38.996866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.997327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:38.997327 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:38.997327 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:38.997327 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:38.997327 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:38.997327 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:38.997327 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.997327 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:38.998216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.998216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:38.998216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:38.998216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:38.998216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.998216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:35:38.998216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:38.998216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:38.998216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:38.998216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:38.998216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:38.998216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:38.998561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:38.998561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:38.998561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:38.998561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:38.998561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:38.998561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:38.998561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:38.998561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:38.998561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:38.998561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:38.998561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:38.998561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:38.998561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:38.998929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.998929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:38.999271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:38.999271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:38.999271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:38.999271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:38.999271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:38.999271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:38.999271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.999271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:38.999271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:38.999271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.999271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:38.999271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:38.999271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.999271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:38.999271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:38.999271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:38.999747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:35:38.999747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:38.999747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:38.999747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:38.999747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:38.999747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:38.999747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:38.999747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:38.999747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:38.999747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:38.999747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:38.999747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:38.999747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:38.999747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:38.999747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:38.999747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:38.999747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:38.999747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:38.999747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:39.000232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:39.000232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:39.000232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:43.701912 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:43.703762 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:43.703762 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:43.703762 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:43.737496 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:43.737496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.737810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:43.737810 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:43.737810 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:43.737810 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:43.737810 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:43.737810 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:43.737810 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.737810 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:43.738662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.738662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:43.738662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:43.738662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:43.738662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:43.738662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:35:43.738662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:43.738662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:43.738662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:43.738662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:43.738662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:43.738662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:43.739271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:43.739271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:43.739271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:43.739271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:43.739271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:43.739271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:43.739271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:43.739271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:43.739271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:43.739271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:43.739271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:43.739271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:43.739271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:43.739931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.739931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:43.739931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.739931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:43.739931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:43.739931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:43.739931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:43.739931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:43.739931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:43.739931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:43.739931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:43.739931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:43.739931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:43.739931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:43.739931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:43.740652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:43.740652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:43.740652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:43.740652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:35:43.740652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:43.740652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:43.740652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:43.740652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:43.740652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:43.740652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:43.740652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:43.740652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:43.740652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:43.740652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:43.740652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:43.740652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:43.740652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:43.740652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:43.740652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:43.741746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:43.741746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:43.741746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:43.741746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:43.741746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.741746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:43.932964 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:43.933555 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:43.933800 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:43.936010 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:43.936010 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:43.936010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.936360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:43.936360 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:43.936360 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:43.936360 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:43.936360 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:43.936360 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:43.936360 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.936360 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:43.937721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.937721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:43.937721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:43.937721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:43.937721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:43.937721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:35:43.937721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:43.937721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:43.937721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:43.937721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:43.937721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:43.937721 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:43.938279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:43.938279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:43.938279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:43.938279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:43.938279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:43.938279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:43.938279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:43.938279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:43.938279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:43.938279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:43.938279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:43.938279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:43.938279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:43.938625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.938625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:43.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:43.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:43.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:43.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:43.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:43.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:43.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:43.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:43.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:43.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:43.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:43.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:43.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:43.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:43.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:43.939890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:35:43.939890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:43.939890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:43.939890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:43.939890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:43.939890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:43.939890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:43.939890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:43.939890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:43.939890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:43.939890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:43.939890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:43.939890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:43.939890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:43.939890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:43.939890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:43.939890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:43.939890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:43.939890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:43.940404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:43.940404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.940404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:43.977614 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:43.978004 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:43.978235 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:43.978425 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:35:43.979800 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:43.979800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.980770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:43.980770 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:43.980770 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:43.980770 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:43.980770 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:43.980770 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:43.980770 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.980770 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:43.982117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.982117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:35:43.982117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:43.982117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:43.982117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:43.982117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:43.982117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:43.982117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:43.982117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:43.982117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:43.982117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:43.982117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:35:43.982117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:43.982117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:35:43.982117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:35:43.983113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:43.983113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:35:43.983113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:43.983113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:35:43.983113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:35:43.983113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:43.983113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:35:43.983113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.983113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:35:43.983523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.983523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:43.983523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:43.983523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:43.983523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:43.983523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:35:43.983523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:35:43.983523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:35:43.983523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:43.983523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:43.983523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:43.983523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:43.984071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:43.984071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:43.984071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:43.984071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:43.984071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:43.984071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:43.984071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:43.984071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:43.984071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:43.984071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:43.984071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:43.984071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:43.984071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:43.984071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:43.984655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:43.984655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:43.984655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.984655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:43.985640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.985640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:43.985640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:43.985640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:43.985640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:43.985640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:43.985640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:43.985640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:43.985640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:43.985640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:43.985640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:43.985640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:43.985640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:43.985640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:43.985640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:43.985640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:43.987089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:35:43.987089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:35:43.987089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:35:43.987089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:43.987089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:43.987089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:43.987089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:43.987089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:43.987089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:43.987089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:43.987089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:43.987089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:43.987089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:43.987089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:43.987089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:43.987089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:43.987089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:43.987089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:43.988776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:43.988776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:43.988776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:43.988776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:43.988776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:43.988776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.988776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:43.988776 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:43.988776 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:43.988776 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:43.989716 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:43.989716 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:43.989716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.989716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:43.989716 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:43.989716 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:43.989716 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:43.989716 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:43.989716 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:43.989716 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.989716 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:43.989716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:35:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:35:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:35:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:35:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:35:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:43.991003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:35:43.991003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:35:43.991003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:43.991003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:35:43.991003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.991003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:35:43.991003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.991003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:43.991003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:43.991003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:43.991003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:43.991003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:35:43.991003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:35:43.991511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:43.991511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:43.991511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:43.991511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:43.991511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:43.991511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:43.991511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:43.991511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:43.991511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:43.991511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:43.991511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:43.991511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:43.991511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:43.992034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:43.992034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:43.992034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:43.992034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:43.992034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:43.992034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:43.992034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:43.992034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.992034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:43.992034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.992034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:43.992034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:43.992284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:43.992284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:43.992284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:43.992284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:43.992284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:43.992284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:43.992284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:43.992284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:43.992284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:43.992284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:43.992284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:43.992284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:43.992284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:43.992284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:35:43.992284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:35:43.992284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:43.992284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:43.992284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:43.992284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:43.992612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:43.992612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:43.992612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:43.992612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:43.992612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:43.992612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:43.992612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:43.992612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:43.992612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:43.992612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:43.992612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:43.992612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:43.992612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:43.992612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:43.992612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:43.992612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:43.992612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.992612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:43.996052 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:43.996375 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:43.996491 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:43.996645 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:43.997116 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:43.997472 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:43.997729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.998419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:43.998419 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:43.998419 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:43.998419 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:43.998419 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:43.998419 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:43.998419 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.998419 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:43.999311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:43.999311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:43.999311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:43.999311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:43.999311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:43.999311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:35:43.999311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:43.999311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:43.999311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:43.999311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:43.999311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:43.999311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:43.999655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:43.999655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:43.999655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:43.999655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:43.999655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:43.999655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:43.999655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:43.999655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:43.999655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:43.999655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:43.999655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:43.999655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:43.999655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:44.000051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:44.000051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:44.000333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:44.000333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:44.000333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:44.000333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:44.000333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:44.000333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:44.000333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:44.000333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:44.000333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:44.000333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:44.000333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:44.000333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:44.000333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:44.000333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:44.000333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:44.000333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:44.000763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:35:44.000763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:44.000763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:44.000763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:44.000763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:44.000763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:44.000763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:44.000763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:44.000763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:44.000763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:44.000763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:44.000763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:44.000763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:44.000763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:44.000763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:44.000763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:44.000763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:44.000763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:44.000763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:44.001263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:44.001263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:44.001263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:48.702500 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:48.703355 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:48.703355 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:48.704502 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:48.739869 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:48.740128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.740871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:48.740871 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:48.740871 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:48.740871 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:48.740871 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:48.740871 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:48.740871 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.740871 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:48.742028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.742028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:48.742028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:48.742028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:48.742028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:48.742028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:35:48.742028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:48.742028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:48.742028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:48.742028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:48.742028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:48.742028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:48.742951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:48.742951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:48.742951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:48.742951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:48.742951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:48.742951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:48.742951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:48.742951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:48.742951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:48.742951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:48.742951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:48.742951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:48.742951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:48.743864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.743864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:48.743864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.743864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:48.743864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:48.743864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:48.743864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:48.743864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:48.743864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:48.743864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:48.743864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:48.743864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:48.743864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:48.743864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:48.743864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:48.745038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:48.745038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:48.745038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:48.745038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:35:48.745038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:48.745038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:48.745038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:48.745038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:48.745038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:48.745038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:48.745038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:48.745038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:48.745038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:48.745038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:48.745038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:48.745038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:48.745038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:48.745038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:48.745038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:48.745960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:48.745960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:48.745960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:48.745960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:48.745960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.745960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:48.933745 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:48.934261 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:48.934450 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:48.935563 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:48.935903 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:48.936253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.937187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:48.937187 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:48.937187 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:48.937187 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:48.937187 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:48.937187 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:48.937187 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.937187 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:48.938502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.938502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:48.938502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:48.938502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:48.938502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:48.938502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:35:48.938502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:48.938502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:48.938502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:48.938502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:48.938502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:48.938502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:48.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:48.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:48.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:48.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:48.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:48.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:48.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:48.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:48.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:48.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:48.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:48.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:48.939362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:48.940173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.940173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:48.940315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.940315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:48.940315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:48.940315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:48.940315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:48.940315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:48.940315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:48.940315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:48.940315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:48.940315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:48.940315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:48.940315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:48.940315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:48.940315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:48.940315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:48.940315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:48.941272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:35:48.941272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:48.941272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:48.941272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:48.941272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:48.941272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:48.941272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:48.941272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:48.941272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:48.941272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:48.941272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:48.941272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:48.941272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:48.941272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:48.941272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:48.941272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:48.941272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:48.941272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:48.941272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:48.942421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:48.942421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.942421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:48.978355 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:48.978594 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:48.978790 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:48.978985 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:35:48.979706 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:48.980013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.980917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:48.980917 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:48.980917 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:48.980917 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:48.980917 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:48.980917 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:48.980917 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.980917 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:48.982097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.982097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:35:48.982097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:48.982097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:48.982097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:48.982097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:48.982097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:48.982097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:48.982097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:48.982097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:48.982097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:48.982097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:35:48.982097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:48.982097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:35:48.982097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:35:48.983233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:48.983233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:35:48.983233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:48.983233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:35:48.983233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:35:48.983233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:48.983233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:35:48.983233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.983233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:35:48.983891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.983891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:48.983891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:48.983891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:48.983891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:48.983891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:35:48.983891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:35:48.983891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:35:48.983891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:48.983891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:48.983891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:48.983891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:48.984762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:48.984762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:48.984762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:48.984762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:48.984762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:48.984762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:48.984762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:48.984762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:48.984762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:48.984762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:48.984762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:48.984762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:48.984762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:48.984762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:48.985456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:48.985456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:48.985456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.985456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:48.985456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.985456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:48.985456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:48.985456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:48.985456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:48.985456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:48.985456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:48.985456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:48.985456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:48.985456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:48.986047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:48.986047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:48.986047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:48.986047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:48.986047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:48.986047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:48.986047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:35:48.986047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:35:48.986047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:35:48.986047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:48.986047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:48.986047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:48.986047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:48.986047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:48.986047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:48.986047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:48.986047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:48.986047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:48.986047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:48.986775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:48.986775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:48.986775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:48.986775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:48.986775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:48.986775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:48.986775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:48.986775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:48.986775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:48.986775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:48.986775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.986775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:48.987191 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:48.987293 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:48.987470 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:48.987660 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:48.988200 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:48.988558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.989334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:48.989334 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:48.989334 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:48.989334 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:48.989334 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:48.989334 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:48.989334 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.989334 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:48.990092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.990092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:35:48.990092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:48.990092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:48.990092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:48.990092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:48.990092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:48.990092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:48.990092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:48.990092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:48.990092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:48.990092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:35:48.990092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:48.990092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:35:48.990092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:35:48.990626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:48.990626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:35:48.990626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:48.990626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:35:48.990626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:35:48.990626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:48.990626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:35:48.990626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.990626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:35:48.991039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.991039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:48.991039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:48.991039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:48.991039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:48.991039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:35:48.991039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:35:48.991039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:48.991039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:48.991039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:48.991039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:48.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:48.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:48.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:48.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:48.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:48.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:48.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:48.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:48.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:48.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:48.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:48.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:48.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:48.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:48.991996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:48.991996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:48.991996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.991996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:48.992183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.992183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:48.992183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:48.992183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:48.992183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:48.992183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:48.992183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:48.992183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:48.992183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:48.992183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:48.992183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:48.992183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:48.992183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:48.992183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:48.992183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:48.992183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:48.992183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:35:48.992879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:35:48.992879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:48.992879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:48.992879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:48.992879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:48.992879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:48.992879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:48.992879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:48.992879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:48.992879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:48.992879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:48.992879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:48.992879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:48.992879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:48.992879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:48.992879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:48.992879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:48.992879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:48.992879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:48.993553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:48.993553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:48.993553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.993553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:48.996453 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:48.996747 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:48.996859 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:48.997033 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:48.997421 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:48.997817 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:48.998032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.998659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:48.998659 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:48.998659 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:48.998659 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:48.998659 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:48.998659 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:48.998659 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.998659 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:48.999523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:48.999523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:48.999523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:48.999523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:48.999523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:48.999523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:35:48.999523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:48.999523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:48.999523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:48.999523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:48.999523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:48.999523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:49.000111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:49.000111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:49.000111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:49.000111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:49.000111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:49.000111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:49.000111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:49.000111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:49.000111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:49.000111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:49.000111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:49.000111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:49.000111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:49.000548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:49.000548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:49.000548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:49.000548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:49.000548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:49.000548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:49.000548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:49.000548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:49.000548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:49.000548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:49.000548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:49.000548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:49.000548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:49.000548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:49.000548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:49.001100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:49.001100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:49.001100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:49.001100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:35:49.001100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:49.001100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:49.001100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:49.001100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:49.001100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:49.001100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:49.001100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:49.001100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:49.001100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:49.001100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:49.001100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:49.001100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:49.001100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:49.001100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:49.001100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:49.001794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:49.001794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:49.001794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:49.001794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:49.001794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:49.001794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:53.703354 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:53.704145 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:53.704145 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:53.704780 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:53.746724 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:53.747114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.748041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:53.748041 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:53.748041 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:53.748041 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:53.748041 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:53.748041 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:53.748041 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.748041 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:53.748841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.748841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:53.748841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:53.748841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:53.748841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:53.748841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:35:53.748841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:53.748841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:53.748841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:53.748841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:53.748841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:53.748841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:53.749441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:53.749441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:53.749441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:53.749441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:53.749441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:53.749441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:53.749441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:53.749441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:53.749441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:53.749441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:53.749441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:53.749441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:53.749441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:53.750089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.750089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:53.750089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.750089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:53.750089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:53.750089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:53.750089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:53.750089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:53.750089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:53.750089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:53.750089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:53.750089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:53.750089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:53.750089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:53.750089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:53.750878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:53.750878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:53.750878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:53.750878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:35:53.750878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:53.750878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:53.750878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:53.750878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:53.750878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:53.750878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:53.750878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:53.750878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:53.750878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:53.750878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:53.750878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:53.750878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:53.750878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:53.750878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:53.750878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:53.752473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:53.752473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:53.752473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:53.752473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:53.752473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.752473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:53.934638 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:53.935261 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:53.935457 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:53.936655 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:53.937041 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:53.937310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.938489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:53.938489 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:53.938489 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:53.938489 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:53.938489 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:53.938489 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:53.938489 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.938489 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:53.940340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.940340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:53.940340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:53.940340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:53.940340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:53.940340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:35:53.940340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:53.940340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:53.940340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:53.940340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:53.940340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:53.940340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:53.940567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:53.940567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:53.940567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:53.940567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:53.940567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:53.940567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:53.940567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:53.940567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:53.940567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:53.940567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:53.940567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:53.940567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:53.940567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:53.940838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.940838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:53.942258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.942258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:53.942258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:53.942258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:53.942258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:53.942258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:53.942258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:53.942258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:53.942258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:53.942258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:53.942258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:53.942258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:53.942258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:53.942258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:53.942258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:53.942258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:53.942553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:35:53.942553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:53.942553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:53.942553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:53.942553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:53.942553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:53.942553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:53.942553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:53.942553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:53.942553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:53.942553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:53.942553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:53.942553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:53.942553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:53.942553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:53.942553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:53.942553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:53.942553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:53.942553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:53.942942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:53.942942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.942942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:53.978762 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:53.979190 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:53.979318 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:53.979446 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:35:53.980071 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:53.980385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.981294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:53.981294 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:53.981294 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:53.981294 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:53.981294 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:53.981294 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:53.981294 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.981294 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:53.982485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.982485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:35:53.982485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:53.982485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:53.982485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:53.982485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:53.982485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:53.982485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:53.982485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:53.982485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:53.982485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:53.982485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:35:53.982485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:53.982485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:35:53.982485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:35:53.982790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:53.982790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:35:53.982790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:53.982790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:35:53.982790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:35:53.982790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:53.982790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:35:53.982790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.982790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:35:53.983824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.983824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:53.983824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:53.983824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:53.983824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:53.983824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:35:53.983824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:35:53.983824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:35:53.983824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:53.983824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:53.983824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:53.983824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:53.984069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:53.984069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:53.984069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:53.984069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:53.984069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:53.984069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:53.984069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:53.984069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:53.984069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:53.984069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:53.984069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:53.984069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:53.984069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:53.984069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:53.984330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:53.984330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:53.984330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.984330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:53.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:53.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:53.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:53.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:53.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:53.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:53.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:53.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:53.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:53.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:53.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:53.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:53.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:53.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:53.985524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:53.986478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:35:53.986478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:35:53.986478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:35:53.986478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:53.986478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:53.986478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:53.986478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:53.986478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:53.986478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:53.986478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:53.986478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:53.986478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:53.986478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:53.986478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:53.986478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:53.986478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:53.986478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:53.986478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:53.987357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:53.987357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:53.987357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:53.987357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:53.987357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:53.987357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.987357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:53.987357 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:53.987744 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:53.988309 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:53.988309 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:53.988593 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:53.989081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.989812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:53.989812 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:53.989812 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:53.989812 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:53.989812 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:53.989812 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:53.989812 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.989812 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:53.990568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.990568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:35:53.990568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:53.990568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:53.990568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:53.990568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:53.990568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:53.990568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:53.990568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:53.990568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:53.990568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:53.990568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:35:53.990568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:53.990568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:35:53.990568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:35:53.991646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:53.991646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:35:53.991646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:53.991646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:35:53.991646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:35:53.991646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:53.991646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:35:53.991646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.991646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:35:53.991646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.991646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:53.991646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:53.991646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:53.991646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:53.992318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:35:53.992318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:35:53.992318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:53.992318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:53.992318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:53.992318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:53.992318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:53.992318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:53.992318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:53.992318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:53.992318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:53.992318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:53.992318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:53.992895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:53.992895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:53.992895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:53.992895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:53.992895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:53.992895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:53.992895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:53.992895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:53.992895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:53.992895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.992895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:53.992895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.993469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:53.993469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:53.993469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:53.993469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:53.993469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:53.993469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:53.993469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:53.993469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:53.993469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:53.993469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:53.993469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:53.993469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:53.993469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:53.993469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:53.993469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:53.993469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:35:53.993469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:35:53.993469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:53.993469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:53.993972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:53.993972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:53.993972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:53.993972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:53.993972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:53.993972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:53.993972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:53.993972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:53.993972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:53.993972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:53.993972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:53.993972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:53.993972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:53.993972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:53.993972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:53.993972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:53.993972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:53.993972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:53.993972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:53.993972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:53.996933 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:53.997197 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:53.997312 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:53.997464 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:53.998895 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:53.999248 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:53.999480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:54.000216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:54.000216 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:54.000216 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:54.000216 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:54.000216 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:54.000216 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:54.000216 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:54.000216 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:54.001982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:54.001982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:54.001982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:54.001982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:54.001982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:54.001982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:35:54.001982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:54.001982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:54.001982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:54.001982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:54.001982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:54.001982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:54.002819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:54.002819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:54.002819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:54.002819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:54.002819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:54.002819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:54.002819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:54.002819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:54.002819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:54.002819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:54.002819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:54.002819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:54.002819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:54.003623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:54.003623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:54.003623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:54.003623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:54.003623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:54.003623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:54.003623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:54.003623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:54.003623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:54.003623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:54.003623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:54.003623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:54.003623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:54.003623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:54.003623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:54.004389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:54.004389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:54.004389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:54.004389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:35:54.004389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:54.004389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:54.004389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:54.004389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:54.004389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:54.004389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:54.004389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:54.004389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:54.004389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:54.004389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:54.004389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:54.004389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:54.004389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:54.004389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:54.004389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:54.005281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:54.005281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:54.005281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:54.005281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:54.005281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:54.005281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:58.704035 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:58.704541 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:58.704881 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:58.706275 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:58.746386 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:58.748178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.748548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:58.748548 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:58.748548 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:58.748548 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:58.748548 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:58.748548 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:58.748548 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.748548 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:58.749617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.749617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:58.749617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:58.749617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:58.749617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:58.749617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:35:58.749617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:58.749617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:58.749617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:58.749617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:58.749617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:58.749617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:58.750965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:58.750965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:58.750965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:58.750965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:58.750965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:58.750965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:58.750965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:58.750965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:58.750965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:58.750965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:58.750965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:58.750965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:58.750965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:58.751683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.751683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:58.751683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.751683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:58.751683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:58.751683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:58.751683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:58.751683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:58.751683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:58.751683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:58.751683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:58.751683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:58.751683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:58.751683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:58.751683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:58.752598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:58.752598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:58.752598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:58.752598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:35:58.752598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:58.752598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:58.752598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:58.752598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:58.752598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:58.752598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:58.752598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:58.752598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:58.752598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:58.752598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:58.752598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:58.752598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:58.752598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:58.752598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:58.752598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:58.753885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:58.753885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:58.753885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:58.753885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:58.753885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.753885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:58.935493 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:58.936252 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:58.936450 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:58.937666 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:58.937976 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:58.938165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.938938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:58.938938 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:58.938938 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:58.938938 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:58.938938 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:58.938938 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:58.938938 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.938938 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:58.940632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.940632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:58.940632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:58.940632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:58.940632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:58.940632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:35:58.940632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:58.940632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:58.940632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:58.940632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:58.940632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:58.940632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:58.941287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:58.941287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:58.941287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:58.941287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:58.941287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:58.941287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:58.941287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:58.941287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:58.941287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:58.941287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:58.941287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:58.941287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:58.941287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:58.941849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.941849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:58.942345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.942345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:58.942345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:58.942345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:58.942345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:58.942345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:58.942345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:58.942345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:58.942345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:58.942345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:58.942345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:58.942345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:58.942345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:58.942345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:58.942345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:58.942345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:58.943060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:35:58.943060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:58.943060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:58.943060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:58.943060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:58.943060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:58.943060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:58.943060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:58.943060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:58.943060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:58.943060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:58.943060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:58.943060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:58.943060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:58.943060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:58.943060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:58.943060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:58.943060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:58.943060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:58.943924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:58.943924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.943924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:58.979577 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:58.980067 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:58.980250 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:58.980432 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:35:58.981034 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:58.981399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.982502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:58.982502 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:58.982502 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:58.982502 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:58.982502 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:58.982502 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:58.982502 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.982502 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:58.983708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.983708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:35:58.983708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:58.983708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:58.983708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:58.983708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:58.983708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:58.983708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:58.983708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:58.983708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:58.983708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:58.983708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:35:58.983708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:58.983708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:35:58.983708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:35:58.984490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:58.984490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:35:58.984490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:58.984490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:35:58.984490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:35:58.984490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:58.984490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:35:58.984490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.984490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:35:58.985071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.985071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:58.985071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:58.985071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:58.985071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:58.985071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:35:58.985071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:35:58.985071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:35:58.985071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:58.985071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:58.985071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:58.985071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:58.985654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:58.985654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:58.985654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:58.985654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:58.985654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:58.985654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:58.985654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:58.985654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:58.985654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:58.985654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:58.985654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:58.985654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:58.985654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:58.985654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:58.986251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:58.986251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:58.986251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.986251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:58.986962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.986962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:58.986962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:58.986962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:58.986962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:58.986962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:58.986962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:58.986962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:58.986962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:58.986962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:58.986962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:58.986962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:58.986962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:58.986962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:58.986962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:58.986962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:58.987683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:35:58.987683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:35:58.987683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:35:58.987683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:58.987683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:58.987683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:58.987683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:58.987683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:58.987683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:58.987683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:58.987683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:58.987683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:58.987683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:58.987683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:58.987683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:58.987683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:58.987683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:58.987683 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:58.988560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:58.988560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:58.988560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:58.988560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:58.988560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:58.988560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.988560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:58.988560 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:58.988560 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:58.988560 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:58.989077 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:58.989077 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:58.989281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.989868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:58.989868 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:58.989868 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:58.989868 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:58.989868 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:58.989868 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:58.989868 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.989868 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:58.991401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.991401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:35:58.991401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:58.991401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:58.991401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:58.991401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:58.991401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:58.991401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:58.991401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:58.991401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:35:58.991401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:58.991401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:35:58.991401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:58.991401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:35:58.991401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:35:58.992093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:58.992093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:35:58.992093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:58.992093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:35:58.992093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:35:58.992093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:58.992093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:35:58.992093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.992093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:35:58.993241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.993241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:58.993241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:58.993241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:58.993241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:58.993241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:35:58.993241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:35:58.993241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:58.993241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:58.993241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:58.993241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:58.993603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:58.993603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:58.993603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:58.993603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:58.993603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:58.993603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:58.993603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:58.993603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:58.993603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:58.993603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:58.993603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:58.993603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:58.993603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:58.993603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:58.994020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:58.994020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:58.994020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.994020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:58.994310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.994310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:58.994310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:58.994310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:58.994310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:58.994310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:58.994310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:58.994310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:58.994310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:58.994310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:58.994310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:58.994310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:58.994310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:58.994310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:58.994310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:58.994310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:58.994310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:35:58.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:35:58.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:35:58.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:35:58.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:58.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:58.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:58.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:58.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:58.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:58.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:58.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:58.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:58.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:58.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:58.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:58.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:58.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:58.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:58.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:58.995383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:58.995383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:58.995383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.995383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:35:58.997304 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:35:58.997616 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:35:58.997753 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:35:58.997903 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:35:58.998248 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:58.998523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:35:58.998697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.999233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:35:58.999233 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:58.999233 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:35:58.999233 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:35:58.999233 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:58.999233 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:35:58.999233 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:58.999233 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:35:59.000073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:59.000073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:35:59.000073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:59.000073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:35:59.000073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:59.000073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:35:59.000073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:59.000073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:59.000073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:59.000073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:59.000073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:59.000073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:59.000503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:35:59.000503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:59.000503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:35:59.000503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:59.000503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:35:59.000503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:35:59.000503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:59.000503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:35:59.000503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:59.000503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:35:59.000503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:35:59.000503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:59.000503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:35:59.000872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:59.000872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:35:59.001112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:59.001112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:35:59.001112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:35:59.001112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:35:59.001112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:35:59.001112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:35:59.001112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:59.001112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:35:59.001112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:35:59.001112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:59.001112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:35:59.001112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:35:59.001112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:35:59.001112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:35:59.001112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:35:59.001112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:35:59.001554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:35:59.001554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:35:59.001554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:59.001554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:59.001554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:35:59.001554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:35:59.001554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:35:59.001554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:35:59.001554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:35:59.001554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:35:59.001554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:35:59.001554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:35:59.001554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:35:59.001554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:35:59.001554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:35:59.001554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:35:59.001554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:35:59.001554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:35:59.001554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:35:59.002067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:35:59.002067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:35:59.002067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:03.704479 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:03.705254 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:03.705254 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:03.706095 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:03.739429 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:03.739594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.740075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:03.740075 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:03.740075 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:03.740075 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:03.740075 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:03.740075 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:03.740075 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.740075 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:03.740737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.740737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:03.740737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:03.740737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:03.740737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:03.740737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:36:03.740737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:03.740737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:03.740737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:03.740737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:03.740737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:03.740737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:03.741364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:03.741364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:03.741364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:03.741364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:03.741364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:03.741364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:03.741364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:03.741364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:03.741364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:03.741364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:03.741364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:03.741364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:03.741364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:03.742106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.742106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:03.742415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.742415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:03.742415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:03.742415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:03.742415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:03.742415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:03.742415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:03.742415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:03.742415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:03.742415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:03.742415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:03.742415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:03.742415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:03.742415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:03.742415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:03.742415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:03.743261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:36:03.743261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:03.743261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:03.743261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:03.743261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:03.743261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:03.743261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:03.743261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:03.743261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:03.743261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:03.743261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:03.743261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:03.743261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:03.743261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:03.743261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:03.743261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:03.743261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:03.743261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:03.743261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:03.744544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:03.744544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.744544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:03.936273 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:03.937073 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:03.937073 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:03.937876 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:03.938055 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:03.938186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.938869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:03.938869 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:03.938869 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:03.938869 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:03.938869 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:03.938869 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:03.938869 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.938869 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:03.940433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.940433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:03.940433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:03.940433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:03.940433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:03.940433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:36:03.940433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:03.940433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:03.940433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:03.940433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:03.940433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:03.940433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:03.940901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:03.940901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:03.940901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:03.940901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:03.940901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:03.940901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:03.940901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:03.940901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:03.940901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:03.940901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:03.940901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:03.940901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:03.940901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:03.941380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.941380 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:03.941666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.941666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:03.941666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:03.941666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:03.941666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:03.941666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:03.941666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:03.941666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:03.941666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:03.941666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:03.941666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:03.941666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:03.941666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:03.941666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:03.941666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:03.941666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:03.942175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:36:03.942175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:03.942175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:03.942175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:03.942175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:03.942175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:03.942175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:03.942175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:03.942175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:03.942175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:03.942175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:03.942175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:03.942175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:03.942175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:03.942175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:03.942175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:03.942175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:03.942175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:03.942175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:03.942754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:03.942754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.942754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:03.980210 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:03.980762 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:03.980929 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:03.981087 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:36:03.981474 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:03.981698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:03.983246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.983246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:36:03.983246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:03.983246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:03.983246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:03.983246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:03.983246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:03.983246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:03.983246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:03.983246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:03.983246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:03.983246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:36:03.983246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:03.983246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:36:03.983246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:36:03.984009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:03.984009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:36:03.984009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:03.984009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:36:03.984009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:36:03.984009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:03.984009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:36:03.984009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.984009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:36:03.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:03.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:03.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:03.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:03.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:36:03.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:36:03.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:36:03.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:03.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:03.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:03.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:03.985133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:03.985133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:03.985133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:03.985133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:03.985133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:03.985133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:03.985133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:03.985133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:03.985133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:03.985133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:03.985133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:03.985133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:03.985133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:03.985133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:03.985760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:03.985760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:03.985760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.985760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:03.985760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.985760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:03.985760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:03.985760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:03.985760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:03.985760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:03.985760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:03.985760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:03.985760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:03.985760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:03.986394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:03.986394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:03.986394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:03.986394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:03.986394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:03.986394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:03.986394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:36:03.986394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:36:03.986394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:36:03.986394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:03.986394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:03.986394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:03.986394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:03.986394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:03.986394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:03.986394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:03.986394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:03.986394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:03.986394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:03.987499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:03.987499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:03.987499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:03.987499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:03.987499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:03.987499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:03.987499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:03.987499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:03.987499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:03.987499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:03.987499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.987499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:03.988120 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:03.988364 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:03.988466 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:03.988599 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:03.989093 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:03.989523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.990218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:03.990218 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:03.990218 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:03.990218 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:03.990218 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:03.990218 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:03.990218 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.990218 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:03.990929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.990929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:36:03.990929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:03.990929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:03.990929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:03.990929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:03.990929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:03.990929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:03.990929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:03.990929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:03.990929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:03.990929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:36:03.990929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:03.990929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:36:03.990929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:36:03.991328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:03.991328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:36:03.991328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:03.991328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:36:03.991328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:36:03.991328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:03.991328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:36:03.991328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.991328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:36:03.991809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.991809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:03.991809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:03.991809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:03.991809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:03.991809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:36:03.991809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:36:03.991809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:03.991809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:03.991809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:03.991809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:03.992122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:03.992122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:03.992122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:03.992122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:03.992122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:03.992122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:03.992122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:03.992122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:03.992122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:03.992122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:03.992122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:03.992122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:03.992122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:03.992122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:03.992493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:03.992493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:03.992493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.992493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:03.993101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.993101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:03.993101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:03.993101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:03.993101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:03.993101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:03.993101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:03.993101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:03.993101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:03.993101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:03.993101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:03.993101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:03.993101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:03.993101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:03.993101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:03.993101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:03.993101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:36:03.993558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:36:03.993558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:03.993558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:03.993558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:03.993558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:03.993558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:03.993558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:03.993558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:03.993558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:03.993558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:03.993558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:03.993558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:03.993558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:03.993558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:03.993558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:03.993558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:03.993558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:03.993558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:03.993558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:03.994085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:03.994085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:03.994085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.994085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:03.997810 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:03.998092 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:03.998200 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:03.998352 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:03.998620 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:03.998934 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:03.999000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.999391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:03.999391 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:03.999391 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:03.999391 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:03.999391 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:03.999391 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:03.999391 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:03.999391 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:04.000173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:04.000173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:04.000173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:04.000173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:04.000173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:04.000173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:36:04.000173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:04.000173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:04.000173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:04.000173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:04.000173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:04.000173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:04.000506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:04.000506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:04.000506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:04.000506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:04.000506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:04.000506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:04.000506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:04.000506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:04.000506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:04.000506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:04.000506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:04.000506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:04.000506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:04.000876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:04.000876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:04.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:04.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:04.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:04.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:04.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:04.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:04.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:04.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:04.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:04.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:04.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:04.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:04.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:04.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:04.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:04.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:04.001641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:36:04.001641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:04.001641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:04.001641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:04.001641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:04.001641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:04.001641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:04.001641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:04.001641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:04.001641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:04.001641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:04.001641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:04.001641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:04.001641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:04.001641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:04.001641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:04.001641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:04.001641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:04.001641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:04.002182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:04.002182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:04.002182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:08.705123 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:08.705889 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:08.705963 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:08.707196 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:08.736657 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=239 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:36:08.745838 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:08.746382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.747243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:08.747243 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:08.747243 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:08.747243 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:08.747243 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:08.747243 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:08.747243 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.747243 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:08.748116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.748116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:08.748116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:08.748116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:08.748116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:08.748116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:36:08.748116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:08.748116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:08.748116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:08.748116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:08.748116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:08.748116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:08.748529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:08.748529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:08.748529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:08.748529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:08.748529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:08.748529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:08.748529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:08.748529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:08.748529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:08.748529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:08.748529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:08.748529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:08.748529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:08.749036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.749036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:08.749211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.749211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:08.749211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:08.749211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:08.749211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:08.749211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:08.749211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:08.749211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:08.749211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:08.749211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:08.749211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:08.749211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:08.749211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:08.749211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:08.749211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:08.749211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:08.749588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:36:08.749588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:08.749588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:08.749588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:08.749588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:08.749588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:08.749588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:08.749588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:08.749588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:08.749588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:08.749588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:08.749588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:08.749588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:08.749588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:08.749588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:08.749588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:08.749588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:08.749588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:08.749588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:08.750151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:08.750151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.750151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:08.936964 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:08.937505 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:08.937814 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:08.939014 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:08.939373 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:08.939639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.940579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:08.940579 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:08.940579 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:08.940579 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:08.940579 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:08.940579 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:08.940579 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.940579 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:08.942348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.942348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:08.942348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:08.942348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:08.942348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:08.942348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:36:08.942348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:08.942348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:08.942348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:08.942348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:08.942348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:08.942348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:08.943283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:08.943283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:08.943283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:08.943283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:08.943283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:08.943283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:08.943283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:08.943283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:08.943283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:08.943283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:08.943283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:08.943283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:08.943283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:08.944239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.944239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:08.944830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.944830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:08.944830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:08.944830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:08.944830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:08.944830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:08.944830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:08.944830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:08.944830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:08.944830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:08.944830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:08.944830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:08.944830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:08.944830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:08.944830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:08.944830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:08.945331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:36:08.945331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:08.945331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:08.945331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:08.945331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:08.945331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:08.945331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:08.945331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:08.945331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:08.945331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:08.945331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:08.945331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:08.945331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:08.945331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:08.945331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:08.945331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:08.945331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:08.945331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:08.945331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:08.945873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:08.945873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.945873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:08.980842 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:08.981274 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:08.981392 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:08.981520 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:36:08.982268 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:08.982589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.983391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:08.983391 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:08.983391 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:08.983391 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:08.983391 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:08.983391 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:08.983391 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.983391 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:08.984399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.984399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:36:08.984399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:08.984399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:08.984399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:08.984399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:08.984399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:08.984399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:08.984399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:08.984399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:08.984399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:08.984399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:36:08.984399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:08.984399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:36:08.984399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:36:08.985382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:08.985382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:36:08.985382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:08.985382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:36:08.985382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:36:08.985382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:08.985382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:36:08.985382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.985382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:36:08.985382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.985382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:08.985382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:08.985382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:08.985382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:08.986198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:36:08.986198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:36:08.986198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:36:08.986198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:08.986198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:08.986198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:08.986198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:08.986198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:08.986198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:08.986198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:08.986198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:08.986198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:08.986198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:08.986859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:08.986859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:08.986859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:08.986859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:08.986859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:08.986859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:08.986859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:08.986859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:08.986859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:08.986859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:08.986859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.986859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:08.987415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.987415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:08.987415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:08.987415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:08.987415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:08.987415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:08.987415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:08.987415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:08.987415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:08.987415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:08.987415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:08.987415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:08.987415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:08.987415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:08.987415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:08.987415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:08.988162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:36:08.988162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:36:08.988162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:36:08.988162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:08.988162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:08.988162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:08.988162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:08.988162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:08.988162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:08.988162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:08.988162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:08.988162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:08.988162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:08.988162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:08.988162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:08.988162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:08.988162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:08.988162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:08.988956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:08.988956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:08.988956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:08.988956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:08.988956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:08.988956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.988956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:08.988956 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:08.988956 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:08.988956 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:08.989456 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:08.989543 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:08.990052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.990812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:08.990812 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:08.990812 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:08.990812 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:08.990812 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:08.990812 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:08.990812 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.990812 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:08.991579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.991579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:36:08.991579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:08.991579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:08.991579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:08.991579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:08.991579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:08.991579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:08.991579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:08.991579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:08.991579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:08.991579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:36:08.991579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:08.991579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:36:08.991579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:36:08.992302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:08.992302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:36:08.992302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:08.992302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:36:08.992302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:36:08.992302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:08.992302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:36:08.992302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.992302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:36:08.992703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.992703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:08.992703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:08.992703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:08.992703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:08.992703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:36:08.992703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:36:08.992703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:08.992703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:08.992703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:08.992703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:08.993164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:08.993164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:08.993164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:08.993164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:08.993164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:08.993164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:08.993164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:08.993164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:08.993164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:08.993164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:08.993164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:08.993164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:08.993164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:08.993164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:08.993816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:08.993816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:08.993816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.993816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:08.993816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.993816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:08.993816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:08.993816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:08.993816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:08.993816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:08.993816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:08.993816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:08.993816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:08.993816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:08.994355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:08.994355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:08.994355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:08.994355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:08.994355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:08.994355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:08.994355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:36:08.994355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:36:08.994355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:08.994355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:08.994355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:08.994355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:08.994355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:08.994355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:08.994355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:08.994355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:08.994355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:08.994355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:08.994355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:08.995122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:08.995122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:08.995122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:08.995122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:08.995122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:08.995122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:08.995122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:08.995122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:08.995122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:08.995122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:08.995122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:08.998179 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:08.998411 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:08.998514 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:08.998660 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:08.999191 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:08.999565 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:08.999941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:09.000747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:09.000747 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:09.000747 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:09.000747 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:09.000747 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:09.000747 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:09.000747 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:09.000747 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:09.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:09.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:09.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:09.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:09.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:09.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:36:09.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:09.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:09.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:09.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:09.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:09.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:09.002264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:09.002264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:09.002264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:09.002264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:09.002264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:09.002264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:09.002264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:09.002264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:09.002264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:09.002264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:09.002264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:09.002264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:09.002264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:09.002862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:09.002862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:09.002862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:09.002862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:09.002862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:09.002862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:09.002862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:09.002862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:09.002862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:09.002862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:09.002862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:09.002862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:09.002862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:09.002862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:09.002862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:09.003517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:09.003517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:09.003517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:09.003517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:36:09.003517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:09.003517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:09.003517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:09.003517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:09.003517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:09.003517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:09.003517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:09.003517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:09.003517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:09.003517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:09.003517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:09.003517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:09.003517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:09.003517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:09.003517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:09.004519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:09.004519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:09.004519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:09.004519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:09.004519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:09.004519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:13.056284 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:13.057154 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:13.057154 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:13.066934 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
Apr 28 20:36:13.067066 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=106506) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
Apr 28 20:36:13.067193 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 7ff08466-af6d-4e13-bb1d-8987170a5734 (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:36:13.100144 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-7ff08466-af6d-4e13-bb1d-8987170a5734, resource={'id': '7ff08466-af6d-4e13-bb1d-8987170a5734', 'name': 'private', 'tenant_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'project_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['4b461bd9-09ea-4f87-a007-a11ab022ff7f', '8f3677a2-e2be-443a-a90c-2cd7b3095c49'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-28T20:10:58Z', 'updated_at': '2026-04-28T20:11:02Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 46342}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.100484 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-7ff08466-af6d-4e13-bb1d-8987170a5734, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.101045 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-27fbc209-4b05-4b4a-8d5c-40d6d87b7e22, external_ids_update=None, columns={'tag': 46342}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.101260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.102063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:13.102063 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.102063 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:13.102063 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.102063 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.102063 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:13.102063 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.102063 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:13.102889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 7ff08466-af6d-4e13-bb1d-8987170a5734 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.102889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:13.102889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:36:13.102889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:36:13.102889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:36:13.102889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:36:13.102889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:36:13.102889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:13.102889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.102889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:13.102889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:13.103438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:13.103438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.103438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:13.103438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:13.103438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:13.103438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:13.103438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:13.103438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:13.103438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:13.103438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:13.103438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.103438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:13.103438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.103438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.104043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:13.104043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.104043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:13.104043 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 8a1fb774-5693-4139-8b75-6ab5273228c7 (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:36:13.133641 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, resource={'id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'name': 'public', 'tenant_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'project_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['335d12ce-2d0c-4086-b3f4-2af390c03d70', '5308c10f-28d1-441c-acb2-d1d6dc3d2ece'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-04-28T20:11:05Z', 'updated_at': '2026-04-28T20:11:13Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.134069 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.134863 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-7dbf713e-d963-4884-ae64-da4ce196e719, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.135792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.135904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:13.135904 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.135904 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:13.135904 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.135904 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.135904 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:13.135904 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.135904 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:13.136937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 8a1fb774-5693-4139-8b75-6ab5273228c7 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.136937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:13.136937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:36:13.136937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:36:13.136937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:36:13.136937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:36:13.136937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:36:13.136937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:13.136937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.136937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:13.136937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:13.137494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:13.137494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.137494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:13.137494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:13.137494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:13.137494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:13.137494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:13.137494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:13.137494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:13.137494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:13.137494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.137494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:13.137494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.137494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.138177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:13.138177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.138177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:13.138177 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource c6718637-daa9-4db1-bd11-63abe3891db6 (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:36:13.172108 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-c6718637-daa9-4db1-bd11-63abe3891db6, resource={'id': 'c6718637-daa9-4db1-bd11-63abe3891db6', 'name': 'admin_net', 'tenant_id': '3eb96b75a3044dd0a338616e1d9081d0', 'project_id': '3eb96b75a3044dd0a338616e1d9081d0', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['c1cc9883-a7e2-4abc-8892-a7c1b200d3ad'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-28T20:15:13Z', 'updated_at': '2026-04-28T20:15:14Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 7626}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.172456 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-c6718637-daa9-4db1-bd11-63abe3891db6, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.173005 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-7461d8df-a8a0-40c3-87a1-26ed88f9f7ce, external_ids_update=None, columns={'tag': 7626}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.173183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.173913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:13.173913 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.173913 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:13.173913 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.173913 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.173913 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:13.173913 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.173913 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:13.174748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource c6718637-daa9-4db1-bd11-63abe3891db6 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.174748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:13.174748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:36:13.174748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:36:13.174748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:36:13.174748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:36:13.174748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:36:13.174748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:13.174748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.174748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:13.174748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:13.175249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:13.175249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.175249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:13.175249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:13.175249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:13.175249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:13.175249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:13.175249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:13.175249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:13.175249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:13.175249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.175249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:13.175249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.175249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.175862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:13.175862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.175862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:13.175862 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource d3dfebbd-7943-4813-b509-739084ada63b (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:36:13.199219 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d3dfebbd-7943-4813-b509-739084ada63b, resource={'id': 'd3dfebbd-7943-4813-b509-739084ada63b', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-pi48i', 'tenant_id': 'a11fcd8844504e7eaf509ece683e8c8f', 'project_id': 'a11fcd8844504e7eaf509ece683e8c8f', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['c68bc19b-c88a-4cd2-bf3a-d847d267d23f'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-28T20:15:49Z', 'updated_at': '2026-04-28T20:15:50Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 58143}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.199528 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-d3dfebbd-7943-4813-b509-739084ada63b, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-pi48i', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.200056 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-ed33b0a1-b0b7-4e33-aa96-92c114a82f5d, external_ids_update=None, columns={'tag': 58143}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.200213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.200907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:13.200907 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.200907 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:13.200907 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.200907 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.200907 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:13.200907 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.200907 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:13.201694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource d3dfebbd-7943-4813-b509-739084ada63b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.201694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:13.201694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:36:13.201694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:36:13.201694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:36:13.201694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:36:13.201694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:36:13.201694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:13.201694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.201694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:13.201694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:13.202290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:13.202290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.202290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:13.202290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:13.202290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:13.202290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:13.202290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:13.202290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:13.202290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:13.202290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:13.202290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.202290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:13.202290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.202290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.202973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:13.202973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.202973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:13.202973 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 6feb526a-13c6-41ac-983d-ea1d2251ff3d (type: routers) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:36:13.258438 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, resource={'id': '6feb526a-13c6-41ac-983d-ea1d2251ff3d', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-pi48i', 'project_id': 'a11fcd8844504e7eaf509ece683e8c8f', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'external_fixed_ips': [{'subnet_id': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece', 'ip_address': '172.24.5.87'}, {'subnet_id': '335d12ce-2d0c-4086-b3f4-2af390c03d70', 'ip_address': '2001:db8::2d8'}], 'enable_snat': True}, 'gw_port_id': '73493630-b0ac-48ec-be98-5b048d2e2ce7', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-pi48i', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'external_fixed_ips': [{'ip_address': '172.24.5.87', 'subnet_id': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece'}, {'ip_address': '2001:db8::2d8', 'subnet_id': '335d12ce-2d0c-4086-b3f4-2af390c03d70'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-04-28T20:15:51Z', 'updated_at': '2026-04-28T20:15:54Z', 'revision_number': 3, 'tenant_id': 'a11fcd8844504e7eaf509ece683e8c8f'}, resource_type=routers, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.258652 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-pi48i', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.258997 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "73493630-b0ac-48ec-be98-5b048d2e2ce7"')], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.259202 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "73493630-b0ac-48ec-be98-5b048d2e2ce7"')], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.259383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.260081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:13.260081 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.260081 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:13.260081 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.260081 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.260081 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:13.260081 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.260081 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:13.260885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Unable to update router 6feb526a-13c6-41ac-983d-ea1d2251ff3d. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.261274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 6feb526a-13c6-41ac-983d-ea1d2251ff3d (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.261274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:13.261274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:36:13.261274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:36:13.261274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:36:13.261274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:36:13.261274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
Apr 28 20:36:13.261274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Apr 28 20:36:13.261274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
Apr 28 20:36:13.261274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Apr 28 20:36:13.261274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Apr 28 20:36:13.262031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Apr 28 20:36:13.262031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
Apr 28 20:36:13.262031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:13.262031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.262031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:13.262031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:13.262031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:13.262031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.262031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:13.262031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:13.262031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:13.262031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:13.262031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:13.262031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:13.262775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:13.262775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:13.262775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.262775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:13.262775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.262775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.262775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:13.262775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.262775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:13.262775 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 73493630-b0ac-48ec-be98-5b048d2e2ce7 (type: router_ports) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:36:13.358374 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-73493630-b0ac-48ec-be98-5b048d2e2ce7, resource={'id': '73493630-b0ac-48ec-be98-5b048d2e2ce7', 'name': '', 'network_id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'project_id': '', 'mac_address': 'fa:16:3e:f8:07:38', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6feb526a-13c6-41ac-983d-ea1d2251ff3d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece', 'ip_address': '172.24.5.87'}, {'subnet_id': '335d12ce-2d0c-4086-b3f4-2af390c03d70', 'ip_address': '2001:db8::2d8'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000171185', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-04-28T20:15:51Z', 'updated_at': '2026-04-28T20:15:54Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.358857 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-73493630-b0ac-48ec-be98-5b048d2e2ce7, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece 335d12ce-2d0c-4086-b3f4-2af390c03d70', 'neutron:network_name': 'neutron-8a1fb774-5693-4139-8b75-6ab5273228c7', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.87/24', '2001:db8::2d8/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.359401 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=73493630-b0ac-48ec-be98-5b048d2e2ce7, lrouter_port=lrp-73493630-b0ac-48ec-be98-5b048d2e2ce7, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.359912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.360691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:13.360691 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.360691 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:13.360691 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.360691 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.360691 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:13.360691 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.360691 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:13.361720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 73493630-b0ac-48ec-be98-5b048d2e2ce7 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.361720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:13.361720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:36:13.361720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:36:13.361720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:36:13.361720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:36:13.361720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1971, in update_router_port
Apr 28 20:36:13.361720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:13.361720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.361720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:13.361720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:13.362316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:13.362316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.362316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:13.362316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:13.362316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:13.362316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:13.362316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:13.362316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:13.362316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:13.362316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:13.362316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.362316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:13.362316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.362316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.363019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:13.363019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.363019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:13.363019 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_for_inconsistencies finished in 0.305 seconds
Apr 28 20:36:13.705781 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:13.706410 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:13.706603 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:13.707854 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:13.742146 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.742468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.743195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:13.743195 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.743195 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:13.743195 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.743195 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.743195 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:13.743195 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.743195 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:13.743837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.743837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:13.743837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:13.743837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:13.743837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:13.743837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:36:13.743837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:13.743837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.743837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:13.743837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:13.743837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:13.743837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.744467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:13.744467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:13.744467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:13.744467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:13.744467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:13.744467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:13.744467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:13.744467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:13.744467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.744467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:13.744467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.744467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.744467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:13.745107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.745107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:13.745107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.745107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:13.745107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:13.745107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:13.745107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:13.745107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:13.745107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:13.745107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:13.745107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:13.745107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:13.745107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:13.745107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:13.745107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:13.745876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:13.745876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:13.745876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:13.745876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:36:13.745876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:13.745876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.745876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:13.745876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:13.745876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:13.745876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.745876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:13.745876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:13.745876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:13.745876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:13.745876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:13.745876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:13.745876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:13.745876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:13.745876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.746679 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:13.746679 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.746679 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.746679 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:13.746679 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.746679 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:13.937659 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:13.938254 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:13.938502 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:13.939614 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.939891 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.940120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.941013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:13.941013 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.941013 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:13.941013 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.941013 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.941013 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:13.941013 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.941013 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:13.942040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.942040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:13.942040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:13.942040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:13.942040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:13.942040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:36:13.942040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:13.942040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.942040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:13.942040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:13.942040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:13.942040 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.942360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:13.942360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:13.942360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:13.942360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:13.942360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:13.942360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:13.942360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:13.942360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:13.942360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.942360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:13.942360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.942360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.942360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:13.942729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.942729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:13.943160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.943160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:13.943160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:13.943160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:13.943160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:13.943160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:13.943160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:13.943160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:13.943160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:13.943160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:13.943160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:13.943160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:13.943160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:13.943160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:13.943160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:13.943160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:13.943589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:36:13.943589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:13.943589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.943589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:13.943589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:13.943589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:13.943589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.943589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:13.943589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:13.943589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:13.943589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:13.943589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:13.943589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:13.943589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:13.943589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:13.943589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.943589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:13.943589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.943589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.944072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:13.944072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.944072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:13.981354 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:13.981806 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:13.981992 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:13.982185 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:36:13.982741 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.983097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.983905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:13.983905 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.983905 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:13.983905 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.983905 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.983905 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:13.983905 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.983905 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:13.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:36:13.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:13.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:13.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:13.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:13.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:13.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:13.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:13.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:36:13.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:13.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:36:13.985141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:36:13.985956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:13.985956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:36:13.985956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.985956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:36:13.985956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.985956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.985956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:36:13.985956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.985956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:36:13.986566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.986566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:13.986566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:13.986566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:13.986566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:13.986566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:36:13.986566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:36:13.986566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:36:13.986566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:13.986566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:13.986566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.986566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:13.987231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:13.987231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:13.987231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.987231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:13.987231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:13.987231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:13.987231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:13.987231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:13.987231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:13.987231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:13.987231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:13.987231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.987231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:13.987231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.987922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.987922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:13.987922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.987922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:13.988214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.988214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:13.988214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:13.988214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:13.988214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:13.988214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:13.988214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:13.988214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:13.988214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:13.988214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:13.988214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:13.988214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:13.988214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:13.988214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:13.988214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:13.988214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:13.989080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:36:13.989080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:36:13.989080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:36:13.989080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:13.989080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:13.989080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.989080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:13.989080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:13.989080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:13.989080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.989080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:13.989080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:13.989080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:13.989080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:13.989080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:13.989080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:13.989080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:13.989080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:13.990188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.990188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:13.990188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.990188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.990188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:13.990188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.990188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:13.990188 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:13.990188 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:13.990188 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:13.990803 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:13.990803 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.990803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.991460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:13.991460 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.991460 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:13.991460 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.991460 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.991460 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:13.991460 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.991460 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:13.992424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.992424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:36:13.992424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:13.992424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:13.992424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.992424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:13.992424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:13.992424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:13.992424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.992424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:13.992424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:13.992424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:36:13.992424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:13.992424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:36:13.992424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:36:13.993517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:13.993517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:36:13.993517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.993517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:36:13.993517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.993517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.993517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:36:13.993517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.993517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:36:13.993517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.993517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:13.993517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:13.993517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:13.993517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:13.994246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:36:13.994246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:36:13.994246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:13.994246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:13.994246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.994246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:13.994246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:13.994246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:13.994246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.994246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:13.994246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:13.994246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:13.994246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:13.994564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:13.994564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:13.994564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:13.994564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:13.994564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.994564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:13.994564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.994564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.994564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:13.994564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.994564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:13.994564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.994824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:13.994824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:13.994824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:13.994824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:13.994824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:13.994824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:13.994824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:13.994824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:13.994824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:13.994824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:13.994824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:13.994824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:13.994824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:13.994824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:13.994824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:13.994824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:36:13.994824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:36:13.994824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:13.994824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:13.995189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.995189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:13.995189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:13.995189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:13.995189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:13.995189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:13.995189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:13.995189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:13.995189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:13.995189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:13.995189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:13.995189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:13.995189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:13.995189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:13.995189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:13.995189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:13.995189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:13.995189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:13.995189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:13.995189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:13.998597 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:13.998982 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:13.999101 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:13.999257 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:13.999472 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.999624 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:13.999743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:14.000084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:14.000084 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:14.000084 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:14.000084 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:14.000084 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:14.000084 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:14.000084 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:14.000084 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:14.000923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:14.000923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:14.000923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:14.000923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:14.000923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:14.000923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:36:14.000923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:14.000923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:14.000923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:14.000923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:14.000923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:14.000923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:14.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:14.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:14.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:14.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:14.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:14.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:14.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:14.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:14.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:14.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:14.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:14.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:14.001200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:14.001511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:14.001511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:14.001991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:14.001991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:14.001991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:14.001991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:14.001991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:14.001991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:14.001991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:14.001991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:14.001991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:14.001991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:14.001991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:14.001991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:14.001991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:14.001991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:14.001991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:14.001991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:14.002408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:36:14.002408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:14.002408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:14.002408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:14.002408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:14.002408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:14.002408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:14.002408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:14.002408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:14.002408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:14.002408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:14.002408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:14.002408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:14.002408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:14.002408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:14.002408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:14.002408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:14.002408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:14.002408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:14.002967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:14.002967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:14.002967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:18.706362 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:18.707234 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:18.707234 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:18.708391 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:18.742166 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:18.742596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.743475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:18.743475 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:18.743475 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:18.743475 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:18.743475 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:18.743475 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:18.743475 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.743475 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:18.744407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.744407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:18.744407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:18.744407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:18.744407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:18.744407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:36:18.744407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:18.744407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:18.744407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:18.744407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:18.744407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:18.744407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:18.745149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:18.745149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:18.745149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:18.745149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:18.745149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:18.745149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:18.745149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:18.745149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:18.745149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:18.745149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:18.745149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:18.745149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:18.745149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:18.745968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.745968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:18.745968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.745968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:18.745968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:18.745968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:18.745968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:18.745968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:18.745968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:18.745968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:18.745968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:18.745968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:18.745968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:18.745968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:18.745968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:18.746814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:18.746814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:18.746814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:18.746814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:36:18.746814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:18.746814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:18.746814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:18.746814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:18.746814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:18.746814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:18.746814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:18.746814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:18.746814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:18.746814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:18.746814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:18.746814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:18.746814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:18.746814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:18.746814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:18.747809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:18.747809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:18.747809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:18.747809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:18.747809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.747809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:18.938334 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:18.939144 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:18.939144 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:18.940222 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:18.940529 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:18.940905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.941662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:18.941662 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:18.941662 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:18.941662 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:18.941662 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:18.941662 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:18.941662 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.941662 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:18.943100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.943100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:18.943100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:18.943100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:18.943100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:18.943100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:36:18.943100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:18.943100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:18.943100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:18.943100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:18.943100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:18.943100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:18.944057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:18.944057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:18.944057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:18.944057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:18.944057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:18.944057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:18.944057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:18.944057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:18.944057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:18.944057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:18.944057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:18.944057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:18.944057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:18.944885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.944885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:18.944885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.944885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:18.944885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:18.944885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:18.944885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:18.944885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:18.944885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:18.944885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:18.944885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:18.944885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:18.944885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:18.944885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:18.944885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:18.945717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:18.945717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:18.945717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:18.945717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:36:18.945717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:18.945717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:18.945717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:18.945717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:18.945717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:18.945717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:18.945717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:18.945717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:18.945717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:18.945717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:18.945717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:18.945717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:18.945717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:18.945717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:18.945717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:18.946602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:18.946602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:18.946602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:18.946602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:18.946602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.946602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:18.982130 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:18.982655 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:18.982890 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:18.983122 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:36:18.983694 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:18.984017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.984805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:18.984805 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:18.984805 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:18.984805 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:18.984805 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:18.984805 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:18.984805 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.984805 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:18.985966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.985966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:36:18.985966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:18.985966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:18.985966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:18.985966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:18.985966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:18.985966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:18.985966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:18.985966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:18.985966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:18.985966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:36:18.985966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:18.985966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:36:18.985966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:36:18.986883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:18.986883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:36:18.986883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:18.986883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:36:18.986883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:36:18.986883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:18.986883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:36:18.986883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.986883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:36:18.987362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.987362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:18.987362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:18.987362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:18.987362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:18.987362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:36:18.987362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:36:18.987362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:36:18.987362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:18.987362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:18.987362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:18.987362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:18.988073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:18.988073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:18.988073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:18.988073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:18.988073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:18.988073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:18.988073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:18.988073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:18.988073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:18.988073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:18.988073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:18.988073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:18.988073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:18.988073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:18.988708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:18.988708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:18.988708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.988708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:18.988899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.988899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:18.988899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:18.988899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:18.988899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:18.988899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:18.988899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:18.988899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:18.988899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:18.988899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:18.988899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:18.988899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:18.988899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:18.988899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:18.988899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:18.988899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:18.990007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:36:18.990007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:36:18.990007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:36:18.990007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:18.990007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:18.990007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:18.990007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:18.990007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:18.990007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:18.990007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:18.990007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:18.990007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:18.990007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:18.990007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:18.990007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:18.990007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:18.990007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:18.990007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:18.990957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:18.990957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:18.990957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:18.990957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:18.990957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:18.990957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.990957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:18.990957 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:18.990957 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:18.990957 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:18.991460 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:18.991460 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:18.991460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.992085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:18.992085 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:18.992085 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:18.992085 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:18.992085 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:18.992085 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:18.992085 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.992085 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:18.992804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.992804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:36:18.992804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:18.992804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:18.992804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:18.992804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:18.992804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:18.992804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:18.992804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:18.992804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:18.992804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:18.992804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:36:18.992804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:18.992804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:36:18.992804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:36:18.993218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:18.993218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:36:18.993218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:18.993218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:36:18.993218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:36:18.993218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:18.993218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:36:18.993218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.993218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:36:18.993637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.993637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:18.993637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:18.993637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:18.993637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:18.993637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:36:18.993637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:36:18.993637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:18.993637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:18.993637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:18.993637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:18.993997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:18.993997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:18.993997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:18.993997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:18.993997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:18.993997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:18.993997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:18.993997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:18.993997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:18.993997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:18.993997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:18.993997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:18.993997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:18.993997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:18.994364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:18.994364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:18.994364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.994364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:18.994703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.994703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:18.994703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:18.994703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:18.994703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:18.994703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:18.994703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:18.994703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:18.994703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:18.994703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:18.994703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:18.994703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:18.994703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:18.994703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:18.994703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:18.994703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:18.994703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:36:18.995176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:36:18.995176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:18.995176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:18.995176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:18.995176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:18.995176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:18.995176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:18.995176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:18.995176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:18.995176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:18.995176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:18.995176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:18.995176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:18.995176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:18.995176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:18.995176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:18.995176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:18.995176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:18.995176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:18.995745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:18.995745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:18.995745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:18.995745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:18.999062 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:18.999306 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:18.999416 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:18.999565 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:19.000053 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:19.000424 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:19.000657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:19.001599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:19.001599 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:19.001599 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:19.001599 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:19.001599 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:19.001599 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:19.001599 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:19.001599 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:19.002251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:19.002251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:19.002251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:19.002251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:19.002251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:19.002251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:36:19.002251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:19.002251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:19.002251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:19.002251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:19.002251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:19.002251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:19.002638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:19.002638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:19.002638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:19.002638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:19.002638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:19.002638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:19.002638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:19.002638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:19.002638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:19.002638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:19.002638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:19.002638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:19.002638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:19.003041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:19.003041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:19.003274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:19.003274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:19.003274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:19.003274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:19.003274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:19.003274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:19.003274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:19.003274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:19.003274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:19.003274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:19.003274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:19.003274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:19.003274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:19.003274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:19.003274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:19.003274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:19.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:36:19.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:19.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:19.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:19.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:19.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:19.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:19.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:19.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:19.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:19.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:19.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:19.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:19.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:19.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:19.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:19.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:19.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:19.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:19.004447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:19.004447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:19.004447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:23.707002 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:23.707727 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:23.707727 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:23.709059 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:23.744496 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:23.744825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.745591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:23.745591 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:23.745591 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:23.745591 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:23.745591 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:23.745591 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:23.745591 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.745591 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:23.746404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.746404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:23.746404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:23.746404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:23.746404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:23.746404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:36:23.746404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:23.746404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:23.746404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:23.746404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:23.746404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:23.746404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:23.747102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:23.747102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:23.747102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:23.747102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:23.747102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:23.747102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:23.747102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:23.747102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:23.747102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:23.747102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:23.747102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:23.747102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:23.747102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:23.747805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.747805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:23.747805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.747805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:23.747805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:23.747805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:23.747805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:23.747805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:23.747805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:23.747805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:23.747805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:23.747805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:23.747805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:23.747805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:23.747805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:23.748546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:23.748546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:23.748546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:23.748546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:36:23.748546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:23.748546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:23.748546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:23.748546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:23.748546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:23.748546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:23.748546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:23.748546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:23.748546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:23.748546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:23.748546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:23.748546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:23.748546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:23.748546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:23.748546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:23.749134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:23.749134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:23.749134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:23.749134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:23.749134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.749134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:23.938952 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:23.939710 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:23.939989 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:23.941526 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:23.941952 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:23.942306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.943445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:23.943445 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:23.943445 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:23.943445 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:23.943445 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:23.943445 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:23.943445 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.943445 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:23.944827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.944827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:23.944827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:23.944827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:23.944827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:23.944827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:36:23.944827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:23.944827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:23.944827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:23.944827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:23.944827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:23.944827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:23.945468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:23.945468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:23.945468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:23.945468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:23.945468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:23.945468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:23.945468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:23.945468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:23.945468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:23.945468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:23.945468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:23.945468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:23.945468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:23.946034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.946034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:23.946034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.946034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:23.946034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:23.946034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:23.946034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:23.946034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:23.946034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:23.946034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:23.946034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:23.946034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:23.946034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:23.946034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:23.946034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:23.946771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:23.946771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:23.946771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:23.946771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:36:23.946771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:23.946771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:23.946771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:23.946771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:23.946771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:23.946771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:23.946771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:23.946771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:23.946771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:23.946771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:23.946771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:23.946771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:23.946771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:23.946771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:23.946771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:23.947609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:23.947609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:23.947609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:23.947609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:23.947609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.947609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:23.982635 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:23.983049 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:23.983176 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:23.983288 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:36:23.983953 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:23.984365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.985320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:23.985320 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:23.985320 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:23.985320 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:23.985320 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:23.985320 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:23.985320 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.985320 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:23.986180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.986180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:36:23.986180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:23.986180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:23.986180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:23.986180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:23.986180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:23.986180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:23.986180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:23.986180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:23.986180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:23.986180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:36:23.986180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:23.986180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:36:23.986180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:36:23.987065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:23.987065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:36:23.987065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:23.987065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:36:23.987065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:36:23.987065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:23.987065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:36:23.987065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.987065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:36:23.987550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.987550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:23.987550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:23.987550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:23.987550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:23.987550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:36:23.987550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:36:23.987550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:36:23.987550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:23.987550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:23.987550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:23.987550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:23.988196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:23.988196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:23.988196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:23.988196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:23.988196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:23.988196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:23.988196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:23.988196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:23.988196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:23.988196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:23.988196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:23.988196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:23.988196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:23.988196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:23.988762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:23.988762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:23.988762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.988762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:23.989519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.989519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:23.989519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:23.989519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:23.989519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:23.989519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:23.989519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:23.989519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:23.989519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:23.989519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:23.989519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:23.989519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:23.989519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:23.989519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:23.989519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:23.989519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:23.990026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:36:23.990026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:36:23.990026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:36:23.990026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:23.990026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:23.990026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:23.990026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:23.990026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:23.990026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:23.990026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:23.990026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:23.990026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:23.990026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:23.990026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:23.990026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:23.990026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:23.990026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:23.990026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:23.990523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:23.990523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:23.990523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:23.990523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:23.990523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:23.990523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.990523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:23.990523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:23.990523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:23.990523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:23.990850 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:23.991201 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:23.991667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.992515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:23.992515 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:23.992515 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:23.992515 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:23.992515 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:23.992515 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:23.992515 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.992515 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:23.993608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.993608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:36:23.993608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:23.993608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:23.993608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:23.993608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:23.993608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:23.993608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:23.993608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:23.993608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:23.993608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:23.993608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:36:23.993608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:23.993608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:36:23.993608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:36:23.994044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:23.994044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:36:23.994044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:23.994044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:36:23.994044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:36:23.994044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:23.994044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:36:23.994044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.994044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:36:23.994473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.994473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:23.994473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:23.994473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:23.994473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:23.994473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:36:23.994473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:36:23.994473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:23.994473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:23.994473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:23.994473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:23.994811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:23.994811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:23.994811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:23.994811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:23.994811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:23.994811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:23.994811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:23.994811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:23.994811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:23.994811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:23.994811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:23.994811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:23.994811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:23.994811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:23.995186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:23.995186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:23.995186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.995186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:23.995608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.995608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:23.995608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:23.995608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:23.995608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:23.995608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:23.995608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:23.995608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:23.995608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:23.995608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:23.995608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:23.995608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:23.995608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:23.995608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:23.995608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:23.995608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:23.995608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:36:23.996126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:36:23.996126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:23.996126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:23.996126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:23.996126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:23.996126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:23.996126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:23.996126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:23.996126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:23.996126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:23.996126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:23.996126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:23.996126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:23.996126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:23.996126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:23.996126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:23.996126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:23.996126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:23.996126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:23.996706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:23.996706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:23.996706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:23.996706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:23.999463 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:23.999772 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:23.999888 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:24.000042 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:24.000603 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:24.001002 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:24.001236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:24.001966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:24.001966 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:24.001966 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:24.001966 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:24.001966 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:24.001966 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:24.001966 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:24.001966 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:24.002770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:24.002770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:24.002770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:24.002770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:24.002770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:24.002770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:36:24.002770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:24.002770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:24.002770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:24.002770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:24.002770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:24.002770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:24.003148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:24.003148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:24.003148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:24.003148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:24.003148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:24.003148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:24.003148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:24.003148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:24.003148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:24.003148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:24.003148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:24.003148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:24.003148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:24.003500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:24.003500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:24.003742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:24.003742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:24.003742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:24.003742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:24.003742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:24.003742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:24.003742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:24.003742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:24.003742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:24.003742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:24.003742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:24.003742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:24.003742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:24.003742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:24.003742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:24.003742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:24.004223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:36:24.004223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:24.004223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:24.004223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:24.004223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:24.004223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:24.004223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:24.004223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:24.004223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:24.004223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:24.004223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:24.004223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:24.004223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:24.004223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:24.004223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:24.004223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:24.004223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:24.004223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:24.004223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:24.004828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:24.004828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:24.004828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:28.707666 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:28.708368 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:28.708526 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:28.709793 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:28.747363 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:28.747749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.748643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:28.748643 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:28.748643 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:28.748643 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:28.748643 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:28.748643 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:28.748643 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.748643 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:28.749779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.749779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:28.749779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:28.749779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:28.749779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:28.749779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:36:28.749779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:28.749779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:28.749779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:28.749779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:28.749779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:28.749779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:28.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:28.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:28.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:28.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:28.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:28.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:28.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:28.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:28.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:28.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:28.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:28.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:28.750363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:28.750979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.750979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:28.750979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.750979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:28.750979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:28.750979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:28.750979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:28.750979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:28.750979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:28.750979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:28.750979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:28.750979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:28.750979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:28.750979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:28.750979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:28.751715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:28.751715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:28.751715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:28.751715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:36:28.751715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:28.751715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:28.751715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:28.751715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:28.751715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:28.751715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:28.751715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:28.751715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:28.751715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:28.751715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:28.751715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:28.751715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:28.751715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:28.751715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:28.751715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:28.752581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:28.752581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:28.752581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:28.752581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:28.752581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.752581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:28.939710 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:28.940386 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:28.940592 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:28.941737 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:28.941986 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:28.942155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.942895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:28.942895 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:28.942895 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:28.942895 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:28.942895 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:28.942895 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:28.942895 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.942895 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:28.944486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.944486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:28.944486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:28.944486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:28.944486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:28.944486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:36:28.944486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:28.944486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:28.944486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:28.944486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:28.944486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:28.944486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:28.945977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:28.945977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:28.945977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:28.945977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:28.945977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:28.945977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:28.945977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:28.945977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:28.945977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:28.945977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:28.945977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:28.945977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:28.945977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:28.947638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.947638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:28.947638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.947638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:28.947638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:28.947638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:28.947638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:28.947638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:28.947638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:28.947638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:28.947638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:28.947638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:28.947638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:28.947638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:28.947638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:28.948818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:28.948818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:28.948818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:28.948818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:36:28.948818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:28.948818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:28.948818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:28.948818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:28.948818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:28.948818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:28.948818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:28.948818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:28.948818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:28.948818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:28.948818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:28.948818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:28.948818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:28.948818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:28.948818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:28.949352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:28.949352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:28.949352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:28.949352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:28.949352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.949352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:28.984227 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:28.984542 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:28.984856 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:28.985119 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:36:28.985667 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:28.986022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.986883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:28.986883 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:28.986883 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:28.986883 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:28.986883 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:28.986883 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:28.986883 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.986883 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:28.988872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.988872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:36:28.988872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:28.988872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:28.988872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:28.988872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:28.988872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:28.988872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:28.988872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:28.988872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:28.988872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:28.988872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:36:28.988872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:28.988872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:36:28.988872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:36:28.989928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:28.989928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:36:28.989928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:28.989928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:36:28.989928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:36:28.989928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:28.989928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:36:28.989928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.989928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:36:28.989928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.989928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:28.989928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:28.989928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:28.989928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:28.990608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:36:28.990608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:36:28.990608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:36:28.990608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:28.990608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:28.990608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:28.990608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:28.990608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:28.990608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:28.990608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:28.990608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:28.990608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:28.990608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:28.991208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:28.991208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:28.991208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:28.991208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:28.991208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:28.991208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:28.991208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:28.991208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:28.991208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:28.991208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:28.991208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.991208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:28.992851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.992851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:28.992851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:28.992851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:28.992851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:28.992851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:28.992851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:28.992851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:28.992851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:28.992851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:28.992851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:28.992851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:28.992851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:28.992851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:28.992851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:28.992851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:28.993898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:36:28.993898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:36:28.993898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:36:28.993898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:28.993898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:28.993898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:28.993898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:28.993898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:28.993898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:28.993898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:28.993898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:28.993898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:28.993898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:28.993898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:28.993898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:28.993898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:28.993898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:28.993898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:28.995086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:28.995086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:28.995086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:28.995086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:28.995086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:28.995086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.995086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:28.995086 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:28.995086 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:28.995086 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:28.995863 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:28.995863 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:28.995863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.995863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:28.995863 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:28.995863 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:28.995863 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:28.995863 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:28.995863 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:28.995863 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.995863 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:28.995863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.996904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:36:28.996904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:28.996904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:28.996904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:28.996904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:28.996904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:28.996904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:28.996904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:28.996904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:28.996904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:28.996904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:36:28.996904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:28.996904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:36:28.996904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:36:28.996904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:28.996904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:36:28.996904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:28.997782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:36:28.997782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:36:28.997782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:28.997782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:36:28.997782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.997782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:36:28.997782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.997782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:28.997782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:28.997782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:28.997782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:28.997782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:36:28.997782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:36:28.998232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:28.998232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:28.998232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:28.998232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:28.998232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:28.998232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:28.998232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:28.998232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:28.998232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:28.998232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:28.998232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:28.998232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:28.998232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:28.998642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:28.998642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:28.998642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:28.998642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:28.998642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:28.998642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:28.998642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:28.998642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.998642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:28.998642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.998642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:28.998642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:28.999073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:28.999073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:28.999073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:28.999073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:28.999073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:28.999073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:28.999073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:28.999073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:28.999073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:28.999073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:28.999073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:28.999073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:28.999073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:28.999073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:36:28.999073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:36:28.999073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:28.999073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:28.999073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:28.999073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:28.999729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:28.999729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:28.999729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:28.999729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:28.999729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:28.999729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:28.999729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:28.999729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:28.999729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:28.999729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:28.999729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:28.999729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:28.999729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:28.999729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:28.999729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:28.999729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:28.999729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:28.999729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:29.000287 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:29.000623 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:29.000718 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:29.000853 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:29.001296 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:29.001622 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:29.001857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:29.002520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:29.002520 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:29.002520 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:29.002520 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:29.002520 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:29.002520 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:29.002520 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:29.002520 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:29.003185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:29.003185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:29.003185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:29.003185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:29.003185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:29.003185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:36:29.003185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:29.003185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:29.003185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:29.003185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:29.003185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:29.003185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:29.003745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:29.003745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:29.003745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:29.003745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:29.003745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:29.003745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:29.003745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:29.003745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:29.003745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:29.003745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:29.003745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:29.003745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:29.003745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:29.004238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:29.004238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:29.004238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:29.004238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:29.004238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:29.004238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:29.004238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:29.004238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:29.004238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:29.004238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:29.004238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:29.004238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:29.004238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:29.004238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:29.004238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:29.005009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:29.005009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:29.005009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:29.005009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:36:29.005009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:29.005009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:29.005009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:29.005009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:29.005009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:29.005009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:29.005009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:29.005009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:29.005009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:29.005009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:29.005009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:29.005009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:29.005009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:29.005009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:29.005009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:29.005783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:29.005783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:29.005783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:29.005783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:29.005783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:29.005783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:33.708386 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:33.709530 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:33.709530 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:33.710906 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:33.739385 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:33.739761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.740453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:33.740453 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:33.740453 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:33.740453 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:33.740453 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:33.740453 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:33.740453 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.740453 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:33.741267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.741267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:33.741267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:33.741267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:33.741267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:33.741267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:36:33.741267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:33.741267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:33.741267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:33.741267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:33.741267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:33.741267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:33.741723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:33.741723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:33.741723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:33.741723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:33.741723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:33.741723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:33.741723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:33.741723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:33.741723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:33.741723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:33.741723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:33.741723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:33.741723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:33.742240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.742240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:33.742476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.742476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:33.742476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:33.742476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:33.742476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:33.742476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:33.742476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:33.742476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:33.742476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:33.742476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:33.742476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:33.742476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:33.742476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:33.742476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:33.742476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:33.742476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:33.742966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:36:33.742966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:33.742966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:33.742966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:33.742966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:33.742966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:33.742966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:33.742966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:33.742966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:33.742966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:33.742966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:33.742966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:33.742966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:33.742966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:33.742966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:33.742966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:33.742966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:33.742966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:33.742966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:33.743547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:33.743547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.743547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:33.940599 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:33.941181 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:33.941402 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:33.942706 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:33.943074 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:33.943318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.944239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:33.944239 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:33.944239 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:33.944239 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:33.944239 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:33.944239 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:33.944239 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.944239 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:33.945818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.945818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:33.945818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:33.945818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:33.945818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:33.945818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:36:33.945818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:33.945818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:33.945818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:33.945818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:33.945818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:33.945818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:33.946424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:33.946424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:33.946424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:33.946424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:33.946424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:33.946424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:33.946424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:33.946424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:33.946424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:33.946424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:33.946424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:33.946424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:33.946424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:33.947185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.947185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:33.947552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.947552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:33.947552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:33.947552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:33.947552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:33.947552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:33.947552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:33.947552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:33.947552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:33.947552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:33.947552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:33.947552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:33.947552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:33.947552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:33.947552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:33.947552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:33.948269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:36:33.948269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:33.948269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:33.948269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:33.948269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:33.948269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:33.948269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:33.948269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:33.948269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:33.948269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:33.948269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:33.948269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:33.948269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:33.948269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:33.948269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:33.948269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:33.948269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:33.948269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:33.948269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:33.949101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:33.949101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.949101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:33.984744 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:33.985108 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:33.985287 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:33.985471 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:36:33.986040 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:33.986358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.987159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:33.987159 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:33.987159 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:33.987159 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:33.987159 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:33.987159 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:33.987159 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.987159 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:33.988401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.988401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:36:33.988401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:33.988401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:33.988401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:33.988401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:33.988401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:33.988401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:33.988401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:33.988401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:33.988401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:33.988401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:36:33.988401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:33.988401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:36:33.988401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:36:33.989199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:33.989199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:36:33.989199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:33.989199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:36:33.989199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:36:33.989199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:33.989199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:36:33.989199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.989199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:36:33.989762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.989762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:33.989762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:33.989762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:33.989762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:33.989762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:36:33.989762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:36:33.989762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:36:33.989762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:33.989762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:33.989762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:33.989762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:33.990442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:33.990442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:33.990442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:33.990442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:33.990442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:33.990442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:33.990442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:33.990442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:33.990442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:33.990442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:33.990442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:33.990442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:33.990442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:33.990442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:33.991045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:33.991045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:33.991045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.991045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:33.991223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.991223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:33.991223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:33.991223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:33.991223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:33.991223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:33.991223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:33.991223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:33.991223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:33.991223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:33.991223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:33.991223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:33.991223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:33.991223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:33.991223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:33.991223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:33.992059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:36:33.992059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:36:33.992059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:36:33.992059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:33.992059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:33.992059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:33.992059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:33.992059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:33.992059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:33.992059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:33.992059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:33.992059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:33.992059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:33.992059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:33.992059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:33.992059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:33.992059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:33.992059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:33.992892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:33.992892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:33.992892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:33.992892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:33.992892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:33.992892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.992892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:33.992892 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:33.992892 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:33.992892 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:33.993351 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:33.993351 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:33.993351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.993742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:33.993742 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:33.993742 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:33.993742 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:33.993742 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:33.993742 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:33.993742 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.993742 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:33.994496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.994496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:36:33.994496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:33.994496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:33.994496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:33.994496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:33.994496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:33.994496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:33.994496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:33.994496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:33.994496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:33.994496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:36:33.994496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:33.994496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:36:33.994496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:36:33.995085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:33.995085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:36:33.995085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:33.995085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:36:33.995085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:36:33.995085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:33.995085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:36:33.995085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.995085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:36:33.995320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.995320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:33.995320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:33.995320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:33.995320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:33.995320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:36:33.995320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:36:33.995320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:33.995320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:33.995320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:33.995320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:33.995652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:33.995652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:33.995652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:33.995652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:33.995652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:33.995652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:33.995652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:33.995652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:33.995652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:33.995652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:33.995652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:33.995652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:33.995652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:33.995652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:33.996022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:33.996022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:33.996022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.996022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:33.996386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.996386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:33.996386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:33.996386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:33.996386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:33.996386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:33.996386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:33.996386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:33.996386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:33.996386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:33.996386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:33.996386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:33.996386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:33.996386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:33.996386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:33.996386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:33.996386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:36:33.997080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:36:33.997080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:33.997080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:33.997080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:33.997080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:33.997080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:33.997080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:33.997080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:33.997080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:33.997080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:33.997080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:33.997080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:33.997080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:33.997080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:33.997080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:33.997080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:33.997080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:33.997080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:33.997080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:33.997644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:33.997644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:33.997644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:33.997644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:34.000835 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:34.001119 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:34.001242 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:34.001387 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:34.001908 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:34.002327 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:34.002548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:34.003299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:34.003299 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:34.003299 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:34.003299 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:34.003299 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:34.003299 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:34.003299 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:34.003299 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:34.004128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:34.004128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:34.004128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:34.004128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:34.004128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:34.004128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:36:34.004128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:34.004128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:34.004128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:34.004128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:34.004128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:34.004128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:34.004478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:34.004478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:34.004478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:34.004478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:34.004478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:34.004478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:34.004478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:34.004478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:34.004478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:34.004478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:34.004478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:34.004478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:34.004478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:34.004843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:34.004843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:34.005088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:34.005088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:34.005088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:34.005088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:34.005088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:34.005088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:34.005088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:34.005088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:34.005088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:34.005088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:34.005088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:34.005088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:34.005088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:34.005088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:34.005088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:34.005088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:34.005589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:36:34.005589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:34.005589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:34.005589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:34.005589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:34.005589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:34.005589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:34.005589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:34.005589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:34.005589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:34.005589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:34.005589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:34.005589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:34.005589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:34.005589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:34.005589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:34.005589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:34.005589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:34.005589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:34.006301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:34.006301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:34.006301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:38.709241 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:38.710127 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:38.710127 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:38.710517 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:38.742579 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:38.743074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.744107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:38.744107 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:38.744107 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:38.744107 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:38.744107 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:38.744107 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:38.744107 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.744107 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:38.745191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.745191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:38.745191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:38.745191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:38.745191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:38.745191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:36:38.745191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:38.745191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:38.745191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:38.745191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:38.745191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:38.745191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:38.745910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:38.745910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:38.745910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:38.745910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:38.745910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:38.745910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:38.745910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:38.745910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:38.745910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:38.745910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:38.745910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:38.745910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:38.745910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:38.746558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.746558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:38.746558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.746558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:38.746558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:38.746558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:38.746558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:38.746558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:38.746558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:38.746558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:38.746558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:38.746558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:38.746558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:38.746558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:38.746558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:38.748294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:38.748294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:38.748294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:38.748294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:36:38.748294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:38.748294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:38.748294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:38.748294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:38.748294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:38.748294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:38.748294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:38.748294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:38.748294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:38.748294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:38.748294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:38.748294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:38.748294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:38.748294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:38.748294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:38.749698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:38.749698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:38.749698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:38.749698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:38.749698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.749698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:38.941324 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:38.941780 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:38.941858 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:38.943141 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:38.943586 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:38.943903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.944914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:38.944914 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:38.944914 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:38.944914 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:38.944914 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:38.944914 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:38.944914 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.944914 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:38.945999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.945999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:38.945999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:38.945999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:38.945999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:38.945999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:36:38.945999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:38.945999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:38.945999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:38.945999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:38.945999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:38.945999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:38.946614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:38.946614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:38.946614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:38.946614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:38.946614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:38.946614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:38.946614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:38.946614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:38.946614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:38.946614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:38.946614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:38.946614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:38.946614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:38.947323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.947323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:38.947323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.947323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:38.947323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:38.947323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:38.947323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:38.947323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:38.947323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:38.947323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:38.947323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:38.947323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:38.947323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:38.947323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:38.947323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:38.948221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:38.948221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:38.948221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:38.948221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:36:38.948221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:38.948221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:38.948221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:38.948221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:38.948221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:38.948221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:38.948221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:38.948221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:38.948221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:38.948221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:38.948221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:38.948221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:38.948221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:38.948221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:38.948221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:38.949457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:38.949457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:38.949457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:38.949457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:38.949457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.949457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:38.985212 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:38.985501 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:38.985601 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:38.985711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:36:38.986347 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:38.986707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.987562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:38.987562 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:38.987562 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:38.987562 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:38.987562 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:38.987562 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:38.987562 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.987562 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:38.988358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.988358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:36:38.988358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:38.988358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:38.988358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:38.988358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:38.988358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:38.988358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:38.988358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:38.988358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:38.988358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:38.988358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:36:38.988358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:38.988358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:36:38.988358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:36:38.989094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:38.989094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:36:38.989094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:38.989094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:36:38.989094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:36:38.989094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:38.989094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:36:38.989094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.989094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:36:38.989523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.989523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:38.989523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:38.989523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:38.989523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:38.989523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:36:38.989523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:36:38.989523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:36:38.989523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:38.989523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:38.989523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:38.989523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:38.990096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:38.990096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:38.990096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:38.990096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:38.990096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:38.990096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:38.990096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:38.990096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:38.990096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:38.990096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:38.990096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:38.990096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:38.990096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:38.990096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:38.990923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:38.990923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:38.990923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.990923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:38.990923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.990923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:38.990923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:38.990923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:38.990923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:38.990923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:38.990923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:38.990923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:38.990923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:38.990923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:38.991604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:38.991604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:38.991604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:38.991604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:38.991604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:38.991604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:38.991604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:36:38.991604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:36:38.991604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:36:38.991604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:38.991604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:38.991604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:38.991604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:38.991604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:38.991604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:38.991604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:38.991604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:38.991604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:38.991604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:38.992975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:38.992975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:38.992975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:38.992975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:38.992975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:38.992975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:38.992975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:38.992975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:38.992975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:38.992975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:38.992975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.992975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:38.992975 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:38.992975 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:38.993632 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:38.993632 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:38.993632 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:38.993632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.994172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:38.994172 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:38.994172 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:38.994172 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:38.994172 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:38.994172 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:38.994172 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.994172 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:38.994935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.994935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:36:38.994935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:38.994935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:38.994935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:38.994935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:38.994935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:38.994935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:38.994935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:38.994935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:38.994935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:38.994935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:36:38.994935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:38.994935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:36:38.994935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:36:38.995351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:38.995351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:36:38.995351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:38.995351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:36:38.995351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:36:38.995351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:38.995351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:36:38.995351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.995351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:36:38.995764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.995764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:38.995764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:38.995764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:38.995764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:38.995764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:36:38.995764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:36:38.995764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:38.995764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:38.995764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:38.995764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:38.996088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:38.996088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:38.996088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:38.996088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:38.996088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:38.996088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:38.996088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:38.996088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:38.996088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:38.996088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:38.996088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:38.996088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:38.996088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:38.996088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:38.996542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:38.996542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:38.996542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.996542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:38.996771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.996771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:38.996771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:38.996771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:38.996771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:38.996771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:38.996771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:38.996771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:38.996771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:38.996771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:38.996771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:38.996771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:38.996771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:38.996771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:38.996771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:38.996771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:38.996771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:36:38.997436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:36:38.997436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:38.997436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:38.997436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:38.997436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:38.997436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:38.997436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:38.997436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:38.997436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:38.997436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:38.997436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:38.997436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:38.997436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:38.997436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:38.997436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:38.997436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:38.997436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:38.997436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:38.997436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:38.997976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:38.997976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:38.997976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:38.997976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:39.001274 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:39.001844 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:39.001844 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:39.001844 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:39.002281 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:39.002728 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:39.003038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:39.003818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:39.003818 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:39.003818 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:39.003818 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:39.003818 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:39.003818 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:39.003818 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:39.003818 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:39.004619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:39.004619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:39.004619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:39.004619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:39.004619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:39.004619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:36:39.004619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:39.004619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:39.004619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:39.004619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:39.004619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:39.004619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:39.005245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:39.005245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:39.005245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:39.005245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:39.005245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:39.005245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:39.005245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:39.005245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:39.005245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:39.005245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:39.005245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:39.005245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:39.005245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:39.005865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:39.005865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:39.005865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:39.005865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:39.005865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:39.005865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:39.005865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:39.005865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:39.005865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:39.005865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:39.005865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:39.005865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:39.005865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:39.005865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:39.005865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:39.006575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:39.006575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:39.006575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:39.006575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:36:39.006575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:39.006575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:39.006575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:39.006575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:39.006575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:39.006575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:39.006575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:39.006575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:39.006575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:39.006575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:39.006575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:39.006575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:39.006575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:39.006575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:39.006575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:39.007475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:39.007475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:39.007475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:39.007475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:39.007475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:39.007475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:43.709843 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:43.710553 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:43.710624 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:43.711896 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:43.743520 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:43.743987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.744916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:43.744916 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:43.744916 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:43.744916 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:43.744916 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:43.744916 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:43.744916 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.744916 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:43.745971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.745971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:43.745971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:43.745971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:43.745971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:43.745971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:36:43.745971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:43.745971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:43.745971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:43.745971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:43.745971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:43.745971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:43.746589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:43.746589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:43.746589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:43.746589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:43.746589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:43.746589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:43.746589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:43.746589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:43.746589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:43.746589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:43.746589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:43.746589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:43.746589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:43.747261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.747261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:43.747261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.747261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:43.747261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:43.747261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:43.747261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:43.747261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:43.747261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:43.747261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:43.747261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:43.747261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:43.747261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:43.747261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:43.747261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:43.748053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:43.748053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:43.748053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:43.748053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:36:43.748053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:43.748053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:43.748053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:43.748053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:43.748053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:43.748053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:43.748053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:43.748053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:43.748053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:43.748053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:43.748053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:43.748053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:43.748053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:43.748053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:43.748053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:43.748992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:43.748992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:43.748992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:43.748992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:43.748992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.748992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:43.941997 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:43.942351 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:43.942531 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:43.943639 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:43.944024 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:43.944243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.945107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:43.945107 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:43.945107 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:43.945107 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:43.945107 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:43.945107 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:43.945107 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.945107 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:43.946493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.946493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:43.946493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:43.946493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:43.946493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:43.946493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:36:43.946493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:43.946493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:43.946493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:43.946493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:43.946493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:43.946493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:43.947257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:43.947257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:43.947257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:43.947257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:43.947257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:43.947257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:43.947257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:43.947257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:43.947257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:43.947257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:43.947257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:43.947257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:43.947257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:43.948013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.948013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:43.948135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.948135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:43.948135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:43.948135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:43.948135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:43.948135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:43.948135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:43.948135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:43.948135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:43.948135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:43.948135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:43.948135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:43.948135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:43.948135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:43.948135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:43.948135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:43.948972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:36:43.948972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:43.948972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:43.948972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:43.948972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:43.948972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:43.948972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:43.948972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:43.948972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:43.948972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:43.948972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:43.948972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:43.948972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:43.948972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:43.948972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:43.948972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:43.948972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:43.948972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:43.948972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:43.949877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:43.949877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.949877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:43.985763 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:43.986080 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:43.986265 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:43.986469 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:36:43.986966 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:43.987248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.988111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:43.988111 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:43.988111 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:43.988111 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:43.988111 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:43.988111 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:43.988111 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.988111 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:43.989320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.989320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:36:43.989320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:43.989320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:43.989320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:43.989320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:43.989320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:43.989320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:43.989320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:43.989320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:43.989320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:43.989320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:36:43.989320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:43.989320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:36:43.989320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:36:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:36:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:36:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:36:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:36:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.990391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:36:43.990984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.990984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:43.990984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:43.990984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:43.990984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:43.990984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:36:43.990984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:36:43.990984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:36:43.990984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:43.990984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:43.990984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:43.990984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:43.991536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:43.991536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:43.991536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:43.991536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:43.991536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:43.991536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:43.991536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:43.991536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:43.991536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:43.991536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:43.991536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:43.991536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:43.991536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:43.991536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:43.992514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:43.992514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:43.992514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.992514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:43.992514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.992514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:43.992514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:43.992514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:43.992514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:43.992514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:43.992514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:43.992514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:43.992514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:43.992514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:43.993294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:43.993294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:43.993294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:43.993294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:43.993294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:43.993294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:43.993294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:36:43.993294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:36:43.993294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:36:43.993294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:43.993294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:43.993294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:43.993294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:43.993294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:43.993294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:43.993294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:43.993294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:43.993294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:43.993294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:43.994343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:43.994343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:43.994343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:43.994343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:43.994343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:43.994343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:43.994343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:43.994343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:43.994343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:43.994343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:43.994343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.994343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:43.994343 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:43.994343 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:43.995177 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:43.995177 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:43.995177 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:43.995177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.995554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:43.995554 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:43.995554 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:43.995554 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:43.995554 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:43.995554 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:43.995554 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.995554 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:43.996761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.996761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:36:43.996761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:43.996761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:43.996761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:43.996761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:43.996761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:43.996761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:43.996761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:43.996761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:43.996761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:43.996761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:36:43.996761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:43.996761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:36:43.996761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:36:43.997158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:43.997158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:36:43.997158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:43.997158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:36:43.997158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:36:43.997158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:43.997158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:36:43.997158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.997158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:36:43.998249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.998249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:43.998249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:43.998249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:43.998249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:43.998249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:36:43.998249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:36:43.998249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:43.998249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:43.998249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:43.998249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:43.998772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:43.998772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:43.998772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:43.998772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:43.998772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:43.998772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:43.998772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:43.998772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:43.998772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:43.998772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:43.998772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:43.998772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:43.998772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:43.998772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:43.999136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:43.999136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:43.999136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:43.999136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:44.000188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:44.000188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:44.000188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:44.000188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:44.000188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:44.000188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:44.000188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:44.000188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:44.000188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:44.000188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:44.000188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:44.000188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:44.000188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:44.000188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:44.000188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:44.000188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:44.000188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:36:44.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:36:44.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:44.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:44.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:44.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:44.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:44.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:44.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:44.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:44.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:44.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:44.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:44.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:44.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:44.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:44.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:44.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:44.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:44.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:44.001357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:44.001357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:44.001357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:44.001357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:44.001828 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:44.002060 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:44.002127 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:44.002241 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:44.002437 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:44.002592 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:44.002703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:44.003035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:44.003035 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:44.003035 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:44.003035 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:44.003035 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:44.003035 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:44.003035 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:44.003035 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:44.003701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:44.003701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:44.003701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:44.003701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:44.003701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:44.003701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:36:44.003701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:44.003701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:44.003701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:44.003701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:44.003701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:44.003701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:44.004099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:44.004099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:44.004099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:44.004099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:44.004099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:44.004099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:44.004099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:44.004099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:44.004099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:44.004099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:44.004099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:44.004099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:44.004099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:44.004460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:44.004460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:44.004460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:44.004460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:44.004460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:44.004460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:44.004460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:44.004460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:44.004460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:44.004460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:44.004460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:44.004460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:44.004460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:44.004460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:44.004460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:44.005007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:44.005007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:44.005007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:44.005007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:36:44.005007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:44.005007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:44.005007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:44.005007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:44.005007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:44.005007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:44.005007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:44.005007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:44.005007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:44.005007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:44.005007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:44.005007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:44.005007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:44.005007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:44.005007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:44.005522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:44.005522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:44.005522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:44.005522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:44.005522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:44.005522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:48.710575 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:48.711086 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:48.711215 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:48.712052 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:48.745410 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:48.745849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.746759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:48.746759 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:48.746759 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:48.746759 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:48.746759 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:48.746759 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:48.746759 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.746759 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:48.747521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.747521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:48.747521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:48.747521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:48.747521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:48.747521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:36:48.747521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:48.747521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:48.747521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:48.747521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:48.747521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:48.747521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:48.747819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:48.747819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:48.747819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:48.747819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:48.747819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:48.747819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:48.747819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:48.747819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:48.747819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:48.747819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:48.747819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:48.747819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:48.747819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:48.748105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.748105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:48.748298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.748298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:48.748298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:48.748298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:48.748298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:48.748298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:48.748298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:48.748298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:48.748298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:48.748298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:48.748298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:48.748298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:48.748298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:48.748298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:48.748298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:48.748298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:48.748640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:36:48.748640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:48.748640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:48.748640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:48.748640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:48.748640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:48.748640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:48.748640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:48.748640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:48.748640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:48.748640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:48.748640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:48.748640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:48.748640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:48.748640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:48.748640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:48.748640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:48.748640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:48.748640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:48.749057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:48.749057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.749057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:48.942760 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:48.943672 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:48.943672 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:48.944378 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:48.944703 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:48.944984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.945805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:48.945805 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:48.945805 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:48.945805 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:48.945805 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:48.945805 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:48.945805 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.945805 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:48.946741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.946741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:48.946741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:48.946741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:48.946741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:48.946741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:36:48.946741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:48.946741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:48.946741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:48.946741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:48.946741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:48.946741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:48.947523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:48.947523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:48.947523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:48.947523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:48.947523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:48.947523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:48.947523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:48.947523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:48.947523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:48.947523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:48.947523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:48.947523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:48.947523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:48.948349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.948349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:48.948349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.948349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:48.948349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:48.948349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:48.948349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:48.948349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:48.948349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:48.948349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:48.948349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:48.948349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:48.948349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:48.948349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:48.948349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:48.949356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:48.949356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:48.949356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:48.949356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:36:48.949356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:48.949356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:48.949356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:48.949356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:48.949356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:48.949356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:48.949356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:48.949356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:48.949356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:48.949356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:48.949356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:48.949356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:48.949356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:48.949356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:48.949356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:48.950708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:48.950708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:48.950708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:48.950708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:48.950708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.950708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:48.986312 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:48.986571 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:48.986673 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:48.986823 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:36:48.987268 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:48.987564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.988389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:48.988389 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:48.988389 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:48.988389 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:48.988389 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:48.988389 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:48.988389 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.988389 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:48.988995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.988995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:36:48.988995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:48.988995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:48.988995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:48.988995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:48.988995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:48.988995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:48.988995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:48.988995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:48.988995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:48.988995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:36:48.988995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:48.988995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:36:48.988995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:36:48.989750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:48.989750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:36:48.989750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:48.989750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:36:48.989750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:36:48.989750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:48.989750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:36:48.989750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.989750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:36:48.990192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.990192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:48.990192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:48.990192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:48.990192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:48.990192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:36:48.990192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:36:48.990192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:36:48.990192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:48.990192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:48.990192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:48.990192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:48.990819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:48.990819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:48.990819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:48.990819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:48.990819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:48.990819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:48.990819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:48.990819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:48.990819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:48.990819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:48.990819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:48.990819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:48.990819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:48.990819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:48.991392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:48.991392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:48.991392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.991392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:48.991392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.991392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:48.991392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:48.991392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:48.991392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:48.991392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:48.991392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:48.991392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:48.991392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:48.991392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:48.992064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:48.992064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:48.992064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:48.992064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:48.992064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:48.992064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:48.992064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:36:48.992064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:36:48.992064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:36:48.992064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:48.992064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:48.992064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:48.992064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:48.992064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:48.992064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:48.992064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:48.992064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:48.992064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:48.992064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:48.992966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:48.992966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:48.992966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:48.992966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:48.992966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:48.992966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:48.992966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:48.992966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:48.992966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:48.992966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:48.992966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.992966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:48.993546 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:48.993546 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:48.993546 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:48.993746 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:48.994124 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:48.994484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.995153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:48.995153 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:48.995153 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:48.995153 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:48.995153 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:48.995153 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:48.995153 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.995153 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:48.995863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.995863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:36:48.995863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:48.995863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:48.995863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:48.995863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:48.995863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:48.995863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:48.995863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:48.995863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:48.995863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:48.995863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:36:48.995863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:48.995863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:36:48.995863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:36:48.996257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:48.996257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:36:48.996257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:48.996257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:36:48.996257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:36:48.996257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:48.996257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:36:48.996257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.996257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:36:48.996728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.996728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:48.996728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:48.996728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:48.996728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:48.996728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:36:48.996728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:36:48.996728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:48.996728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:48.996728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:48.996728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:48.997071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:48.997071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:48.997071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:48.997071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:48.997071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:48.997071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:48.997071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:48.997071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:48.997071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:48.997071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:48.997071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:48.997071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:48.997071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:48.997071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:48.997439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:48.997439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:48.997439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.997439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:48.997805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.997805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:48.997805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:48.997805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:48.997805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:48.997805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:48.997805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:48.997805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:48.997805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:48.997805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:48.997805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:48.997805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:48.997805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:48.997805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:48.997805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:48.997805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:48.997805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:36:48.998299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:36:48.998299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:48.998299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:48.998299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:48.998299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:48.998299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:48.998299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:48.998299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:48.998299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:48.998299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:48.998299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:48.998299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:48.998299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:48.998299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:48.998299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:48.998299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:48.998299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:48.998299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:48.998299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:48.998821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:48.998821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:48.998821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:48.998821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:49.002294 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:49.002585 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:49.002714 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:49.002860 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:49.003348 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:49.003874 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:49.004127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:49.004893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:49.004893 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:49.004893 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:49.004893 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:49.004893 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:49.004893 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:49.004893 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:49.004893 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:49.005779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:49.005779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:49.005779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:49.005779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:49.005779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:49.005779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:36:49.005779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:49.005779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:49.005779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:49.005779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:49.005779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:49.005779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:49.006227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:49.006227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:49.006227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:49.006227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:49.006227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:49.006227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:49.006227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:49.006227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:49.006227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:49.006227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:49.006227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:49.006227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:49.006227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:49.006945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:49.006945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:49.007226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:49.007226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:49.007226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:49.007226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:49.007226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:49.007226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:49.007226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:49.007226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:49.007226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:49.007226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:49.007226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:49.007226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:49.007226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:49.007226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:49.007226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:49.007226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:49.007978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:36:49.007978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:49.007978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:49.007978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:49.007978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:49.007978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:49.007978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:49.007978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:49.007978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:49.007978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:49.007978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:49.007978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:49.007978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:49.007978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:49.007978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:49.007978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:49.007978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:49.007978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:49.007978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:49.008824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:49.008824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:49.008824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:53.711260 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:53.713506 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:53.713850 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:53.715313 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:53.751701 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:53.752124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.753030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:53.753030 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:53.753030 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:53.753030 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:53.753030 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:53.753030 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:53.753030 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.753030 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:53.753933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.753933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:53.753933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:53.753933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:53.753933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:53.753933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:36:53.753933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:53.753933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:53.753933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:53.753933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:53.753933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:53.753933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:53.754336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:53.754336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:53.754336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:53.754336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:53.754336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:53.754336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:53.754336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:53.754336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:53.754336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:53.754336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:53.754336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:53.754336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:53.754336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:53.754762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.754762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:53.754999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.754999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:53.754999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:53.754999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:53.754999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:53.754999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:53.754999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:53.754999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:53.754999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:53.754999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:53.754999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:53.754999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:53.754999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:53.754999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:53.754999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:53.754999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:53.755495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:36:53.755495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:53.755495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:53.755495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:53.755495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:53.755495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:53.755495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:53.755495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:53.755495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:53.755495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:53.755495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:53.755495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:53.755495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:53.755495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:53.755495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:53.755495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:53.755495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:53.755495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:53.755495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:53.756090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:53.756090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.756090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:53.943272 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:53.944009 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:53.944262 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:53.945360 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:53.945701 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:53.945945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.946878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:53.946878 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:53.946878 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:53.946878 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:53.946878 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:53.946878 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:53.946878 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.946878 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:53.948649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.948649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:53.948649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:53.948649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:53.948649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:53.948649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:36:53.948649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:53.948649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:53.948649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:53.948649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:53.948649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:53.948649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:53.949391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:53.949391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:53.949391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:53.949391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:53.949391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:53.949391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:53.949391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:53.949391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:53.949391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:53.949391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:53.949391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:53.949391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:53.949391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:53.950242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.950242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:53.951100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.951100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:53.951100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:53.951100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:53.951100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:53.951100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:53.951100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:53.951100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:53.951100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:53.951100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:53.951100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:53.951100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:53.951100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:53.951100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:53.951100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:53.951100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:53.951821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:36:53.951821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:53.951821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:53.951821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:53.951821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:53.951821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:53.951821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:53.951821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:53.951821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:53.951821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:53.951821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:53.951821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:53.951821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:53.951821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:53.951821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:53.951821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:53.951821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:53.951821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:53.951821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:53.952625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:53.952625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.952625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:53.986711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:53.987215 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:53.987438 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:53.987620 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:36:53.988234 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:53.988571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.989465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:53.989465 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:53.989465 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:53.989465 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:53.989465 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:53.989465 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:53.989465 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.989465 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:53.991389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.991389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:36:53.991389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:53.991389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:53.991389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:53.991389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:53.991389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:53.991389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:53.991389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:53.991389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:53.991389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:53.991389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:36:53.991389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:53.991389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:36:53.991389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:36:53.992217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:53.992217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:36:53.992217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:53.992217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:36:53.992217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:36:53.992217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:53.992217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:36:53.992217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.992217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:36:53.992716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.992716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:53.992716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:53.992716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:53.992716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:53.992716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:36:53.992716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:36:53.992716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:36:53.992716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:53.992716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:53.992716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:53.992716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:53.993327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:53.993327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:53.993327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:53.993327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:53.993327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:53.993327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:53.993327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:53.993327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:53.993327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:53.993327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:53.993327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:53.993327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:53.993327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:53.993327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:53.994135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:53.994135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:53.994135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.994135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:53.994593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.994593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:53.994593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:53.994593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:53.994593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:53.994593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:53.994593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:53.994593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:53.994593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:53.994593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:53.994593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:53.994593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:53.994593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:53.994593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:53.994593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:53.994593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:53.995334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:36:53.995334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:36:53.995334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:36:53.995334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:53.995334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:53.995334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:53.995334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:53.995334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:53.995334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:53.995334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:53.995334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:53.995334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:53.995334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:53.995334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:53.995334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:53.995334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:53.995334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:53.995334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:53.996166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:53.996166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:53.996166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:53.996166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:53.996166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:53.996166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.996166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:53.996166 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:53.996166 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:53.996166 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:53.996631 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:53.996631 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:53.996631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.997481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:53.997481 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:53.997481 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:53.997481 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:53.997481 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:53.997481 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:53.997481 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.997481 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:53.998243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.998243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:36:53.998243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:53.998243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:53.998243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:53.998243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:53.998243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:53.998243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:53.998243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:53.998243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:53.998243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:53.998243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:36:53.998243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:53.998243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:36:53.998243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:36:53.998752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:53.998752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:36:53.998752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:53.998752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:36:53.998752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:36:53.998752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:53.998752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:36:53.998752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.998752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:36:53.999104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.999104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:53.999104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:53.999104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:53.999104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:53.999104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:36:53.999104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:36:53.999104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:53.999104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:53.999104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:53.999104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:53.999393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:53.999393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:53.999393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:53.999393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:53.999393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:53.999393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:53.999393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:53.999393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:53.999393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:53.999393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:53.999393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:53.999393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:53.999393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:53.999393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:53.999659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:53.999659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:53.999659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:53.999659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:54.000205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:54.000205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:54.000205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:54.000205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:54.000205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:54.000205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:54.000205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:54.000205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:54.000205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:54.000205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:54.000205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:54.000205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:54.000205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:54.000205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:54.000205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:54.000205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:54.000205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:36:54.000612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:36:54.000612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:54.000612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:54.000612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:54.000612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:54.000612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:54.000612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:54.000612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:54.000612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:54.000612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:54.000612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:54.000612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:54.000612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:54.000612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:54.000612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:54.000612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:54.000612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:54.000612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:54.000612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:54.001052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:54.001052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:54.001052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:54.001052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:54.002772 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:54.003087 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:54.003209 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:54.003363 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:54.003938 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:54.004318 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:54.004563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:54.005311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:54.005311 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:54.005311 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:54.005311 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:54.005311 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:54.005311 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:54.005311 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:54.005311 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:54.006115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:54.006115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:54.006115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:54.006115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:54.006115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:54.006115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:36:54.006115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:54.006115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:54.006115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:54.006115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:54.006115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:54.006115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:54.006370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:54.006370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:54.006370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:54.006370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:54.006370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:54.006370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:54.006370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:54.006370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:54.006370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:54.006370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:54.006370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:54.006370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:54.006370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:54.006742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:54.006742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:54.007085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:54.007085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:54.007085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:54.007085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:54.007085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:54.007085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:54.007085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:54.007085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:54.007085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:54.007085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:54.007085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:54.007085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:54.007085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:54.007085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:54.007085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:54.007085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:54.007508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:36:54.007508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:54.007508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:54.007508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:54.007508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:54.007508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:54.007508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:54.007508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:54.007508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:54.007508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:54.007508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:54.007508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:54.007508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:54.007508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:54.007508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:54.007508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:54.007508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:54.007508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:54.007508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:54.008052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:54.008052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:54.008052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:58.713398 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:58.714006 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:58.714195 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:58.715394 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:58.748787 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:58.749108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.749852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:58.749852 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:58.749852 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:58.749852 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:58.749852 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:58.749852 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:58.749852 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.749852 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:58.750815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.750815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:58.750815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:58.750815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:58.750815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:58.750815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:36:58.750815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:58.750815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:58.750815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:58.750815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:58.750815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:58.750815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:58.751546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:58.751546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:58.751546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:58.751546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:58.751546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:58.751546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:58.751546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:58.751546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:58.751546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:58.751546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:58.751546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:58.751546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:58.751546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:58.752225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.752225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:58.752225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.752225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:58.752225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:58.752225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:58.752225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:58.752225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:58.752225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:58.752225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:58.752225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:58.752225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:58.752225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:58.752225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:58.752225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:58.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:58.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:58.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:58.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:36:58.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:58.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:58.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:58.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:58.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:58.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:58.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:58.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:58.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:58.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:58.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:58.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:58.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:58.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:58.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:58.754074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:58.754074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:58.754074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:58.754074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:58.754074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.754074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:58.944046 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:58.944832 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:58.944832 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:58.945608 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:58.945924 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:58.946150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.947097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:58.947097 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:58.947097 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:58.947097 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:58.947097 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:58.947097 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:58.947097 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.947097 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:58.948220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.948220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:58.948220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:58.948220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:58.948220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:58.948220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:36:58.948220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:58.948220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:58.948220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:58.948220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:58.948220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:58.948220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:58.948581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:58.948581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:58.948581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:58.948581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:58.948581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:58.948581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:58.948581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:58.948581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:58.948581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:58.948581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:58.948581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:58.948581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:58.948581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:58.948932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.948932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:58.949365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.949365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:58.949365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:58.949365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:58.949365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:58.949365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:58.949365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:58.949365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:58.949365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:58.949365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:58.949365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:58.949365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:58.949365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:58.949365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:58.949365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:58.949365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:58.949868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:36:58.949868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:58.949868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:58.949868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:58.949868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:58.949868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:58.949868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:58.949868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:58.949868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:58.949868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:58.949868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:58.949868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:58.949868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:58.949868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:58.949868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:58.949868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:58.949868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:58.949868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:58.949868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:58.950347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:58.950347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.950347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:58.987164 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:58.987535 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:58.987751 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:58.987943 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:36:58.988425 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:58.988762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.989601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:58.989601 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:58.989601 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:58.989601 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:58.989601 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:58.989601 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:58.989601 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.989601 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:58.990958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.990958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:36:58.990958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:58.990958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:58.990958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:58.990958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:58.990958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:58.990958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:58.990958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:58.990958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:58.990958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:58.990958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:36:58.990958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:58.990958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:36:58.990958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:36:58.991916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:58.991916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:36:58.991916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:58.991916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:36:58.991916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:36:58.991916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:58.991916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:36:58.991916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.991916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:36:58.992345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.992345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:58.992345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:58.992345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:58.992345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:58.992345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:36:58.992345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:36:58.992345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:36:58.992345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:58.992345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:58.992345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:58.992345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:58.992969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:58.992969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:58.992969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:58.992969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:58.992969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:58.992969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:58.992969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:58.992969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:58.992969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:58.992969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:58.992969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:58.992969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:58.992969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:58.992969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:58.993546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:58.993546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:58.993546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.993546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:58.993546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.993546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:58.993546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:58.993546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:58.993546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:58.993546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:58.993546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:58.993546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:58.993546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:58.993546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:58.994175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:58.994175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:58.994175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:58.994175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:58.994175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:58.994175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:58.994175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:36:58.994175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:36:58.994175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:36:58.994175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:58.994175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:58.994175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:58.994175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:58.994175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:58.994175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:58.994175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:58.994175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:58.994175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:58.994175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:58.994993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:58.994993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:58.994993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:58.994993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:58.994993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:58.994993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:58.994993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:58.994993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:58.994993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:58.994993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:58.994993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.994993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:58.995501 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:58.995501 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:58.995619 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:58.995773 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:58.996081 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:58.996293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.996776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:58.996776 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:58.996776 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:58.996776 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:58.996776 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:58.996776 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:58.996776 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.996776 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:58.997479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.997479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:36:58.997479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:58.997479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:58.997479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:58.997479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:58.997479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:58.997479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:58.997479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:58.997479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:36:58.997479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:58.997479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:36:58.997479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:58.997479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:36:58.997479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:36:58.998187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:58.998187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:36:58.998187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:58.998187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:36:58.998187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:36:58.998187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:58.998187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:36:58.998187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.998187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:36:58.998593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.998593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:58.998593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:58.998593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:58.998593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:58.998593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:36:58.998593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:36:58.998593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:58.998593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:58.998593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:58.998593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:58.998931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:58.998931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:58.998931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:58.998931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:58.998931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:58.998931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:58.998931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:58.998931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:58.998931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:58.998931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:58.998931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:58.998931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:58.998931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:58.998931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:58.999318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:58.999318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:58.999318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.999318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:58.999914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:58.999914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:58.999914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:58.999914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:58.999914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:58.999914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:58.999914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:58.999914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:58.999914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:58.999914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:58.999914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:58.999914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:58.999914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:58.999914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:58.999914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:58.999914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:58.999914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:36:59.000371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:36:59.000371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:36:59.000371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:36:59.000371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:59.000371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:59.000371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:59.000371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:59.000371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:59.000371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:59.000371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:59.000371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:59.000371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:59.000371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:59.000371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:59.000371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:59.000371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:59.000371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:59.000371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:59.000371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:59.000907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:59.000907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:59.000907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:59.000907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:36:59.003088 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:36:59.003308 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:36:59.003436 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:36:59.003586 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:36:59.003876 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:59.004086 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:36:59.004221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:59.004738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:36:59.004738 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:59.004738 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:36:59.004738 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:36:59.004738 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:59.004738 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:36:59.004738 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:59.004738 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:36:59.005830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:59.005830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:36:59.005830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:59.005830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:36:59.005830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:59.005830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:36:59.005830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:59.005830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:59.005830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:59.005830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:59.005830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:59.005830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:59.006162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:36:59.006162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:59.006162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:36:59.006162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:59.006162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:36:59.006162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:36:59.006162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:59.006162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:36:59.006162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:59.006162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:36:59.006162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:36:59.006162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:59.006162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:36:59.006590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:59.006590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:36:59.006867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:59.006867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:36:59.006867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:36:59.006867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:36:59.006867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:36:59.006867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:36:59.006867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:59.006867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:36:59.006867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:36:59.006867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:59.006867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:36:59.006867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:36:59.006867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:36:59.006867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:36:59.006867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:36:59.006867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:36:59.007329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:36:59.007329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:36:59.007329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:59.007329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:59.007329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:36:59.007329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:36:59.007329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:36:59.007329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:36:59.007329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:36:59.007329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:36:59.007329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:36:59.007329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:36:59.007329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:36:59.007329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:36:59.007329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:36:59.007329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:36:59.007329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:36:59.007329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:36:59.007329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:36:59.007846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:36:59.007846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:36:59.007846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:03.714041 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:03.714468 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:03.714468 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:03.715499 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:03.750708 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:03.751147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.752043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:03.752043 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:03.752043 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:03.752043 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:03.752043 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:03.752043 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:03.752043 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.752043 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:03.752816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.752816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:03.752816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:03.752816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:03.752816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:03.752816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:37:03.752816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:03.752816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:03.752816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:03.752816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:03.752816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:03.752816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:03.753471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:03.753471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:03.753471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:03.753471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:03.753471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:03.753471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:03.753471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:03.753471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:03.753471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:03.753471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:03.753471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:03.753471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:03.753471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:03.754094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.754094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:03.754094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.754094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:03.754094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:03.754094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:03.754094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:03.754094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:03.754094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:03.754094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:03.754094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:03.754094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:03.754094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:03.754094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:03.754094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:03.754805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:03.754805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:03.754805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:03.754805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:37:03.754805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:03.754805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:03.754805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:03.754805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:03.754805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:03.754805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:03.754805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:03.754805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:03.754805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:03.754805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:03.754805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:03.754805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:03.754805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:03.754805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:03.754805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:03.755588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:03.755588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:03.755588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:03.755588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:03.755588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.755588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:03.944813 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:03.945224 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:03.945469 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:03.946863 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:03.947218 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:03.947456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.948303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:03.948303 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:03.948303 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:03.948303 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:03.948303 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:03.948303 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:03.948303 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.948303 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:03.949476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.949476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:03.949476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:03.949476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:03.949476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:03.949476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:37:03.949476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:03.949476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:03.949476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:03.949476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:03.949476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:03.949476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:03.950205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:03.950205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:03.950205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:03.950205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:03.950205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:03.950205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:03.950205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:03.950205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:03.950205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:03.950205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:03.950205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:03.950205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:03.950205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:03.950850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.950850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:03.950955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.950955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:03.950955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:03.950955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:03.950955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:03.950955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:03.950955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:03.950955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:03.950955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:03.950955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:03.950955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:03.950955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:03.950955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:03.950955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:03.950955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:03.950955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:03.951644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:37:03.951644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:03.951644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:03.951644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:03.951644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:03.951644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:03.951644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:03.951644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:03.951644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:03.951644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:03.951644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:03.951644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:03.951644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:03.951644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:03.951644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:03.951644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:03.951644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:03.951644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:03.951644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:03.952509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:03.952509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.952509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:03.987693 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:03.988133 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:03.988345 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:03.988584 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:37:03.989187 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:03.989486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.990349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:03.990349 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:03.990349 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:03.990349 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:03.990349 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:03.990349 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:03.990349 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.990349 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:03.991821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.991821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:37:03.991821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:03.991821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:03.991821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:03.991821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:03.991821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:03.991821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:03.991821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:03.991821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:03.991821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:03.991821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:37:03.991821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:03.991821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:37:03.991821 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:37:03.992513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:03.992513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:37:03.992513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:03.992513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:37:03.992513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:37:03.992513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:03.992513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:37:03.992513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.992513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:37:03.993520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.993520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:03.993520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:03.993520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:03.993520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:03.993520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:37:03.993520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:37:03.993520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:37:03.993520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:03.993520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:03.993520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:03.993520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:03.994121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:03.994121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:03.994121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:03.994121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:03.994121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:03.994121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:03.994121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:03.994121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:03.994121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:03.994121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:03.994121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:03.994121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:03.994121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:03.994121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:03.994815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:03.994815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:03.994815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.994815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:03.995395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.995395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:03.995395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:03.995395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:03.995395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:03.995395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:03.995395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:03.995395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:03.995395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:03.995395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:03.995395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:03.995395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:03.995395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:03.995395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:03.995395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:03.995395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:03.996184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:37:03.996184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:37:03.996184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:37:03.996184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:03.996184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:03.996184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:03.996184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:03.996184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:03.996184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:03.996184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:03.996184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:03.996184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:03.996184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:03.996184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:03.996184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:03.996184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:03.996184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:03.996184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:03.996977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:03.996977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:03.996977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:03.996977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:03.996977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:03.996977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.996977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:03.996977 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:03.996977 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:03.996977 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:03.997460 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:03.997460 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:03.997460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.997615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:03.997615 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:03.997615 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:03.997615 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:03.997615 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:03.997615 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:03.997615 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.997615 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:03.998500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.998500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:37:03.998500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:03.998500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:03.998500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:03.998500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:03.998500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:03.998500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:03.998500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:03.998500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:03.998500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:03.998500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:37:03.998500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:03.998500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:37:03.998500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:37:03.999191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:03.999191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:37:03.999191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:03.999191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:37:03.999191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:37:03.999191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:03.999191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:37:03.999191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.999191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:37:03.999689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:03.999689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:03.999689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:03.999689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:03.999689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:03.999689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:37:03.999689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:37:03.999689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:03.999689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:03.999689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:03.999689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:04.000224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:04.000224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:04.000224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:04.000224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:04.000224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:04.000224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:04.000224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:04.000224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:04.000224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:04.000224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:04.000224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:04.000224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:04.000224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:04.000224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:04.000888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:04.000888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:04.000888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:04.000888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:04.001218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:04.001218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:04.001218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:04.001218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:04.001218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:04.001218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:04.001218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:04.001218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:04.001218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:04.001218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:04.001218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:04.001218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:04.001218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:04.001218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:04.001218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:04.001218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:04.001218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:37:04.002010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:37:04.002010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:04.002010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:04.002010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:04.002010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:04.002010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:04.002010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:04.002010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:04.002010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:04.002010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:04.002010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:04.002010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:04.002010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:04.002010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:04.002010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:04.002010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:04.002010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:04.002010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:04.002010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:04.002411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:04.002411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:04.002411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:04.002411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:04.003367 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:04.003609 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:04.003782 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:04.003972 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:04.004379 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:04.004744 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:04.004952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:04.005613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:04.005613 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:04.005613 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:04.005613 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:04.005613 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:04.005613 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:04.005613 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:04.005613 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:04.006528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:04.006528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:04.006528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:04.006528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:04.006528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:04.006528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:37:04.006528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:04.006528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:04.006528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:04.006528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:04.006528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:04.006528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:04.006792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:04.006792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:04.006792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:04.006792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:04.006792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:04.006792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:04.006792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:04.006792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:04.006792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:04.006792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:04.006792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:04.006792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:04.006792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:04.007045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:04.007045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:04.007657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:04.007657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:04.007657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:04.007657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:04.007657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:04.007657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:04.007657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:04.007657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:04.007657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:04.007657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:04.007657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:04.007657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:04.007657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:04.007657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:04.007657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:04.007657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:04.007985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:37:04.007985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:04.007985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:04.007985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:04.007985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:04.007985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:04.007985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:04.007985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:04.007985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:04.007985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:04.007985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:04.007985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:04.007985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:04.007985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:04.007985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:04.007985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:04.007985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:04.007985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:04.007985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:04.008328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:04.008328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:04.008328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:08.714714 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:08.715522 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:08.715522 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:08.716720 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:08.742471 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=433 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:37:08.751391 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:08.751877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.752646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:08.752646 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:08.752646 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:08.752646 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:08.752646 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:08.752646 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:08.752646 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.752646 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:08.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:08.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:08.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:08.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:08.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:37:08.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:08.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:08.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:08.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:08.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:08.753545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:08.754156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:08.754156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:08.754156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:08.754156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:08.754156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:08.754156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:08.754156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:08.754156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:08.754156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:08.754156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:08.754156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:08.754156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:08.754156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:08.754761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.754761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:08.754761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.754761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:08.754761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:08.754761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:08.754761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:08.754761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:08.754761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:08.754761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:08.754761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:08.754761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:08.754761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:08.754761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:08.754761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:08.755453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:08.755453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:08.755453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:08.755453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:37:08.755453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:08.755453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:08.755453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:08.755453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:08.755453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:08.755453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:08.755453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:08.755453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:08.755453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:08.755453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:08.755453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:08.755453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:08.755453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:08.755453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:08.755453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:08.756334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:08.756334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:08.756334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:08.756334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:08.756334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.756334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:08.945470 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:08.945988 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:08.946177 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:08.947276 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:08.947596 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:08.948010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.948787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:08.948787 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:08.948787 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:08.948787 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:08.948787 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:08.948787 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:08.948787 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.948787 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:08.950241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.950241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:08.950241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:08.950241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:08.950241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:08.950241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:37:08.950241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:08.950241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:08.950241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:08.950241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:08.950241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:08.950241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:08.950840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:08.950840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:08.950840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:08.950840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:08.950840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:08.950840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:08.950840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:08.950840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:08.950840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:08.950840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:08.950840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:08.950840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:08.950840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:08.951339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.951339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:08.951970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.951970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:08.951970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:08.951970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:08.951970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:08.951970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:08.951970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:08.951970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:08.951970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:08.951970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:08.951970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:08.951970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:08.951970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:08.951970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:08.951970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:08.951970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:08.952670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:37:08.952670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:08.952670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:08.952670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:08.952670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:08.952670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:08.952670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:08.952670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:08.952670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:08.952670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:08.952670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:08.952670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:08.952670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:08.952670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:08.952670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:08.952670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:08.952670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:08.952670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:08.952670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:08.953610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:08.953610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.953610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:08.988268 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:08.988804 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:08.988909 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:08.989013 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:37:08.989776 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:08.990097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.990875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:08.990875 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:08.990875 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:08.990875 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:08.990875 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:08.990875 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:08.990875 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.990875 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:08.991877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.991877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:37:08.991877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:08.991877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:08.991877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:08.991877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:08.991877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:08.991877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:08.991877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:08.991877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:08.991877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:08.991877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:37:08.991877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:08.991877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:37:08.991877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:37:08.992545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:08.992545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:37:08.992545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:08.992545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:37:08.992545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:37:08.992545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:08.992545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:37:08.992545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.992545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:37:08.992888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.992888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:08.992888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:08.992888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:08.992888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:08.992888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:37:08.992888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:37:08.992888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:37:08.992888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:08.992888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:08.992888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:08.992888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:08.993388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:08.993388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:08.993388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:08.993388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:08.993388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:08.993388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:08.993388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:08.993388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:08.993388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:08.993388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:08.993388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:08.993388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:08.993388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:08.993388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:08.993832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:08.993832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:08.993832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.993832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:08.993832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.993832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:08.993832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:08.993832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:08.993832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:08.993832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:08.993832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:08.993832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:08.993832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:08.993832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:08.994279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:08.994279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:08.994279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:08.994279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:08.994279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:08.994279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:08.994279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:37:08.994279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:37:08.994279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:37:08.994279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:08.994279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:08.994279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:08.994279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:08.994279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:08.994279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:08.994279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:08.994279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:08.994279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:08.994279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:08.995403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:08.995403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:08.995403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:08.995403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:08.995403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:08.995403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:08.995403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:08.995403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:08.995403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:08.995403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:08.995403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.995403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:08.996460 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:08.996460 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:08.996460 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:08.996460 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:08.996868 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:08.997111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.997649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:08.997649 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:08.997649 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:08.997649 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:08.997649 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:08.997649 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:08.997649 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.997649 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:08.998353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.998353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:37:08.998353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:08.998353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:08.998353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:08.998353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:08.998353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:08.998353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:08.998353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:08.998353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:08.998353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:08.998353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:37:08.998353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:08.998353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:37:08.998353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:37:08.999158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:08.999158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:37:08.999158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:08.999158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:37:08.999158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:37:08.999158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:08.999158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:37:08.999158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.999158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:37:08.999158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:08.999158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:08.999158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:08.999158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:08.999158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:09.000081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:37:09.000081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:37:09.000081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:09.000081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:09.000081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:09.000081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:09.000081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:09.000081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:09.000081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:09.000081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:09.000081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:09.000081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:09.000081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:09.000605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:09.000605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:09.000605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:09.000605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:09.000605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:09.000605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:09.000605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:09.000605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:09.000605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:09.000605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:09.000605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:09.000605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:09.001015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:09.001015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:09.001015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:09.001015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:09.001015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:09.001015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:09.001015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:09.001015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:09.001015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:09.001015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:09.001015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:09.001015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:09.001015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:09.001015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:09.001015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:09.001015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:37:09.001015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:37:09.001015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:09.001015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:09.001379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:09.001379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:09.001379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:09.001379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:09.001379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:09.001379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:09.001379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:09.001379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:09.001379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:09.001379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:09.001379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:09.001379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:09.001379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:09.001379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:09.001379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:09.001379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:09.001379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:09.001379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:09.001379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:09.001379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:09.003693 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:09.004009 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:09.004097 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:09.004230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:09.004555 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:09.004789 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:09.004935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:09.005481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:09.005481 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:09.005481 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:09.005481 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:09.005481 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:09.005481 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:09.005481 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:09.005481 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:09.006184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:09.006184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:09.006184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:09.006184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:09.006184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:09.006184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:37:09.006184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:09.006184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:09.006184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:09.006184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:09.006184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:09.006184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:09.006650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:09.006650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:09.006650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:09.006650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:09.006650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:09.006650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:09.006650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:09.006650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:09.006650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:09.006650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:09.006650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:09.006650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:09.006650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:09.006910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:09.006910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:09.007083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:09.007083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:09.007083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:09.007083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:09.007083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:09.007083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:09.007083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:09.007083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:09.007083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:09.007083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:09.007083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:09.007083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:09.007083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:09.007083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:09.007083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:09.007083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:09.007396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:37:09.007396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:09.007396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:09.007396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:09.007396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:09.007396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:09.007396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:09.007396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:09.007396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:09.007396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:09.007396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:09.007396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:09.007396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:09.007396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:09.007396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:09.007396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:09.007396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:09.007396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:09.007396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:09.007794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:09.007794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:09.007794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:13.716325 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:13.720289 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:13.720289 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:13.720626 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:13.752224 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:13.752607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:13.753798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:13.753798 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:13.753798 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:13.753798 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:13.753798 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:13.753798 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:13.753798 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:13.753798 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:13.754532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:13.754532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:13.754532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:13.754532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:13.754532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:13.754532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:37:13.754532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:13.754532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:13.754532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:13.754532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:13.754532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:13.754532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:13.755222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:13.755222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:13.755222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:13.755222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:13.755222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:13.755222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:13.755222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:13.755222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:13.755222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:13.755222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:13.755222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:13.755222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:13.755222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:13.755897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:13.755897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:13.756297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:13.756297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:13.756297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:13.756297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:13.756297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:13.756297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:13.756297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:13.756297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:13.756297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:13.756297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:13.756297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:13.756297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:13.756297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:13.756297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:13.756297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:13.756297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:13.757145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:37:13.757145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:13.757145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:13.757145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:13.757145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:13.757145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:13.757145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:13.757145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:13.757145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:13.757145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:13.757145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:13.757145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:13.757145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:13.757145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:13.757145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:13.757145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:13.757145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:13.757145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:13.757145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:13.758136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:13.758136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:13.758136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:13.946052 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:13.946637 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:13.946919 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:13.948359 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:13.948818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:13.949069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:13.950288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:13.950288 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:13.950288 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:13.950288 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:13.950288 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:13.950288 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:13.950288 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:13.950288 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:13.951896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:13.951896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:13.951896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:13.951896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:13.951896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:13.951896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:37:13.951896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:13.951896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:13.951896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:13.951896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:13.951896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:13.951896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:13.952512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:13.952512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:13.952512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:13.952512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:13.952512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:13.952512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:13.952512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:13.952512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:13.952512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:13.952512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:13.952512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:13.952512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:13.952512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:13.953100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:13.953100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:13.953995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:13.953995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:13.953995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:13.953995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:13.953995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:13.953995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:13.953995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:13.953995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:13.953995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:13.953995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:13.953995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:13.953995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:13.953995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:13.953995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:13.953995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:13.953995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:13.955256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:37:13.955256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:13.955256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:13.955256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:13.955256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:13.955256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:13.955256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:13.955256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:13.955256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:13.955256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:13.955256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:13.955256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:13.955256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:13.955256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:13.955256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:13.955256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:13.955256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:13.955256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:13.955256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:13.955803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:13.955803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:13.955803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:13.989141 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:13.989496 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:13.989699 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:13.989890 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:37:13.990418 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:13.990735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:13.991561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:13.991561 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:13.991561 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:13.991561 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:13.991561 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:13.991561 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:13.991561 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:13.991561 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:13.992916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:13.992916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:37:13.992916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:13.992916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:13.992916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:13.992916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:13.992916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:13.992916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:13.992916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:13.992916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:13.992916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:13.992916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:37:13.992916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:13.992916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:37:13.992916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:37:13.993757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:13.993757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:37:13.993757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:13.993757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:37:13.993757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:37:13.993757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:13.993757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:37:13.993757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:13.993757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:37:13.994277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:13.994277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:13.994277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:13.994277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:13.994277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:13.994277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:37:13.994277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:37:13.994277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:37:13.994277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:13.994277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:13.994277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:13.994277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:13.994871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:13.994871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:13.994871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:13.994871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:13.994871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:13.994871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:13.994871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:13.994871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:13.994871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:13.994871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:13.994871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:13.994871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:13.994871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:13.994871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:13.995547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:13.995547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:13.995547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:13.995547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:13.995946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:13.995946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:13.995946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:13.995946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:13.995946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:13.995946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:13.995946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:13.995946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:13.995946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:13.995946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:13.995946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:13.995946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:13.995946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:13.995946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:13.995946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:13.995946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:13.996451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:37:13.996451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:37:13.996451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:37:13.996451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:13.996451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:13.996451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:13.996451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:13.996451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:13.996451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:13.996451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:13.996451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:13.996451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:13.996451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:13.996451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:13.996451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:13.996451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:13.996451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:13.996451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:13.996940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:13.996940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:13.996940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:13.996940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:13.996940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:13.996940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:13.996940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:13.996940 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:13.996940 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:13.996940 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:13.997245 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:13.997611 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:13.997992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:13.998841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:13.998841 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:13.998841 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:13.998841 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:13.998841 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:13.998841 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:13.998841 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:13.998841 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:13.999780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:13.999780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:37:13.999780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:13.999780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:13.999780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:13.999780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:13.999780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:13.999780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:13.999780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:13.999780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:13.999780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:13.999780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:37:13.999780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:13.999780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:37:13.999780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:37:14.000174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:14.000174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:37:14.000174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:14.000174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:37:14.000174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:37:14.000174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:14.000174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:37:14.000174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:14.000174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:37:14.000668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:14.000668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:14.000668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:14.000668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:14.000668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:14.000668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:37:14.000668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:37:14.000668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:14.000668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:14.000668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:14.000668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:14.001011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:14.001011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:14.001011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:14.001011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:14.001011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:14.001011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:14.001011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:14.001011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:14.001011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:14.001011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:14.001011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:14.001011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:14.001011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:14.001011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:14.001398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:14.001398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:14.001398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:14.001398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:14.001745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:14.001745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:14.001745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:14.001745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:14.001745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:14.001745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:14.001745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:14.001745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:14.001745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:14.001745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:14.001745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:14.001745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:14.001745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:14.001745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:14.001745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:14.001745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:14.001745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:37:14.002239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:37:14.002239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:14.002239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:14.002239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:14.002239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:14.002239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:14.002239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:14.002239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:14.002239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:14.002239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:14.002239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:14.002239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:14.002239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:14.002239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:14.002239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:14.002239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:14.002239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:14.002239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:14.002239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:14.002877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:14.002877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:14.002877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:14.002877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:14.004170 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:14.004462 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:14.004575 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:14.004753 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:14.005195 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:14.005698 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:14.006046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:14.006808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:14.006808 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:14.006808 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:14.006808 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:14.006808 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:14.006808 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:14.006808 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:14.006808 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:14.007665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:14.007665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:14.007665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:14.007665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:14.007665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:14.007665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:37:14.007665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:14.007665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:14.007665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:14.007665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:14.007665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:14.007665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:14.008034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:14.008034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:14.008034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:14.008034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:14.008034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:14.008034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:14.008034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:14.008034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:14.008034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:14.008034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:14.008034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:14.008034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:14.008034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:14.008379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:14.008379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:14.008625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:14.008625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:14.008625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:14.008625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:14.008625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:14.008625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:14.008625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:14.008625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:14.008625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:14.008625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:14.008625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:14.008625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:14.008625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:14.008625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:14.008625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:14.008625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:14.009116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:37:14.009116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:14.009116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:14.009116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:14.009116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:14.009116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:14.009116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:14.009116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:14.009116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:14.009116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:14.009116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:14.009116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:14.009116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:14.009116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:14.009116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:14.009116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:14.009116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:14.009116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:14.009116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:14.009584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:14.009584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:14.009584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:18.718438 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:18.719351 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:18.719351 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:18.720452 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:18.753102 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:18.753506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:18.754407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:18.754407 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:18.754407 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:18.754407 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:18.754407 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:18.754407 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:18.754407 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:18.754407 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:18.755238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:18.755238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:18.755238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:18.755238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:18.755238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:18.755238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:37:18.755238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:18.755238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:18.755238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:18.755238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:18.755238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:18.755238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:18.756023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:18.756023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:18.756023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:18.756023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:18.756023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:18.756023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:18.756023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:18.756023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:18.756023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:18.756023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:18.756023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:18.756023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:18.756023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:18.756784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:18.756784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:18.756784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:18.756784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:18.756784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:18.756784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:18.756784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:18.756784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:18.756784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:18.756784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:18.756784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:18.756784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:18.756784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:18.756784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:18.756784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:18.757740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:18.757740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:18.757740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:18.757740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:37:18.757740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:18.757740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:18.757740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:18.757740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:18.757740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:18.757740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:18.757740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:18.757740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:18.757740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:18.757740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:18.757740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:18.757740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:18.757740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:18.757740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:18.757740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:18.758929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:18.758929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:18.758929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:18.758929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:18.758929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:18.758929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:18.946803 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:18.947275 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:18.947462 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:18.948477 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:18.948863 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:18.949097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:18.950038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:18.950038 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:18.950038 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:18.950038 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:18.950038 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:18.950038 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:18.950038 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:18.950038 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:18.951502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:18.951502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:18.951502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:18.951502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:18.951502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:18.951502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:37:18.951502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:18.951502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:18.951502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:18.951502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:18.951502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:18.951502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:18.952298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:18.952298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:18.952298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:18.952298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:18.952298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:18.952298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:18.952298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:18.952298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:18.952298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:18.952298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:18.952298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:18.952298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:18.952298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:18.952866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:18.952866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:18.953152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:18.953152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:18.953152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:18.953152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:18.953152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:18.953152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:18.953152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:18.953152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:18.953152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:18.953152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:18.953152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:18.953152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:18.953152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:18.953152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:18.953152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:18.953152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:18.954243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:37:18.954243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:18.954243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:18.954243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:18.954243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:18.954243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:18.954243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:18.954243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:18.954243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:18.954243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:18.954243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:18.954243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:18.954243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:18.954243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:18.954243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:18.954243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:18.954243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:18.954243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:18.954243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:18.955733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:18.955733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:18.955733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:18.989630 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:18.990050 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:18.990223 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:18.990437 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:37:18.990955 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:18.991267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:18.992148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:18.992148 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:18.992148 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:18.992148 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:18.992148 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:18.992148 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:18.992148 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:18.992148 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:18.993509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:18.993509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:37:18.993509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:18.993509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:18.993509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:18.993509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:18.993509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:18.993509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:18.993509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:18.993509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:18.993509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:18.993509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:37:18.993509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:18.993509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:37:18.993509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:37:18.994336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:18.994336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:37:18.994336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:18.994336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:37:18.994336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:37:18.994336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:18.994336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:37:18.994336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:18.994336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:37:18.994907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:18.994907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:18.994907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:18.994907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:18.994907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:18.994907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:37:18.994907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:37:18.994907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:37:18.994907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:18.994907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:18.994907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:18.994907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:18.995473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:18.995473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:18.995473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:18.995473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:18.995473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:18.995473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:18.995473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:18.995473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:18.995473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:18.995473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:18.995473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:18.995473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:18.995473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:18.995473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:18.996066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:18.996066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:18.996066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:18.996066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:18.996607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:18.996607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:18.996607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:18.996607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:18.996607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:18.996607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:18.996607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:18.996607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:18.996607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:18.996607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:18.996607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:18.996607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:18.996607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:18.996607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:18.996607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:18.996607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:18.997362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:37:18.997362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:37:18.997362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:37:18.997362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:18.997362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:18.997362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:18.997362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:18.997362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:18.997362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:18.997362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:18.997362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:18.997362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:18.997362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:18.997362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:18.997362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:18.997362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:18.997362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:18.997362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:18.998229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:18.998229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:18.998229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:18.998229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:18.998229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:18.998229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:18.998229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:18.998229 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:18.998229 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:18.998229 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:18.998667 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:18.998667 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:18.998773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:18.999518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:18.999518 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:18.999518 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:18.999518 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:18.999518 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:18.999518 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:18.999518 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:18.999518 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:19.000447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:19.000447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:37:19.000447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:19.000447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:19.000447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:19.000447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:19.000447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:19.000447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:19.000447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:19.000447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:19.000447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:19.000447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:37:19.000447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:19.000447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:37:19.000447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:37:19.000914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:19.000914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:37:19.000914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:19.000914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:37:19.000914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:37:19.000914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:19.000914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:37:19.000914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:19.000914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:37:19.001481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:19.001481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:19.001481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:19.001481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:19.001481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:19.001481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:37:19.001481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:37:19.001481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:19.001481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:19.001481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:19.001481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:19.001866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:19.001866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:19.001866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:19.001866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:19.001866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:19.001866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:19.001866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:19.001866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:19.001866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:19.001866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:19.001866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:19.001866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:19.001866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:19.001866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:19.002270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:19.002270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:19.002270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:19.002270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:19.002572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:19.002572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:19.002572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:19.002572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:19.002572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:19.002572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:19.002572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:19.002572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:19.002572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:19.002572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:19.002572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:19.002572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:19.002572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:19.002572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:19.002572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:19.002572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:19.002572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:37:19.003099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:37:19.003099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:19.003099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:19.003099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:19.003099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:19.003099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:19.003099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:19.003099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:19.003099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:19.003099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:19.003099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:19.003099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:19.003099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:19.003099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:19.003099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:19.003099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:19.003099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:19.003099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:19.003099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:19.003644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:19.003644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:19.003644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:19.003644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:19.004638 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:19.004969 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:19.005084 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:19.005235 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:19.005682 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:19.006128 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:19.006353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:19.007098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:19.007098 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:19.007098 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:19.007098 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:19.007098 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:19.007098 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:19.007098 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:19.007098 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:19.007911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:19.007911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:19.007911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:19.007911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:19.007911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:19.007911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:37:19.007911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:19.007911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:19.007911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:19.007911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:19.007911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:19.007911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:19.008287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:19.008287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:19.008287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:19.008287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:19.008287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:19.008287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:19.008287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:19.008287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:19.008287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:19.008287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:19.008287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:19.008287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:19.008287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:19.008632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:19.008632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:19.008846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:19.008846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:19.008846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:19.008846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:19.008846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:19.008846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:19.008846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:19.008846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:19.008846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:19.008846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:19.008846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:19.008846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:19.008846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:19.008846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:19.008846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:19.008846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:19.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:37:19.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:19.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:19.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:19.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:19.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:19.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:19.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:19.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:19.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:19.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:19.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:19.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:19.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:19.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:19.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:19.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:19.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:19.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:19.010029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:19.010029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:19.010029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:23.719081 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:23.719806 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:23.719806 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:23.721018 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:23.753176 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:23.753526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:23.754136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:23.754136 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:23.754136 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:23.754136 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:23.754136 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:23.754136 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:23.754136 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:23.754136 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:23.754871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:23.754871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:23.754871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:23.754871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:23.754871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:23.754871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:37:23.754871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:23.754871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:23.754871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:23.754871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:23.754871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:23.754871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:23.755247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:23.755247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:23.755247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:23.755247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:23.755247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:23.755247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:23.755247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:23.755247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:23.755247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:23.755247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:23.755247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:23.755247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:23.755247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:23.755569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:23.755569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:23.755569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:23.755569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:23.755569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:23.755569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:23.755569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:23.755569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:23.755569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:23.755569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:23.755569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:23.755569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:23.755569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:23.755569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:23.755569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:23.755992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:23.755992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:23.755992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:23.755992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:37:23.755992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:23.755992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:23.755992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:23.755992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:23.755992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:23.755992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:23.755992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:23.755992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:23.755992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:23.755992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:23.755992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:23.755992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:23.755992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:23.755992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:23.755992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:23.756406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:23.756406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:23.756406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:23.756406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:23.756406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:23.756406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:23.947488 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:23.948124 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:23.948308 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:23.949449 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:23.949820 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:23.950069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:23.950992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:23.950992 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:23.950992 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:23.950992 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:23.950992 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:23.950992 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:23.950992 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:23.950992 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:23.952535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:23.952535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:23.952535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:23.952535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:23.952535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:23.952535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:37:23.952535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:23.952535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:23.952535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:23.952535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:23.952535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:23.952535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:23.953307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:23.953307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:23.953307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:23.953307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:23.953307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:23.953307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:23.953307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:23.953307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:23.953307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:23.953307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:23.953307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:23.953307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:23.953307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:23.953860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:23.953860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:23.954383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:23.954383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:23.954383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:23.954383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:23.954383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:23.954383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:23.954383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:23.954383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:23.954383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:23.954383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:23.954383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:23.954383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:23.954383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:23.954383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:23.954383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:23.954383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:23.955111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:37:23.955111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:23.955111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:23.955111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:23.955111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:23.955111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:23.955111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:23.955111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:23.955111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:23.955111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:23.955111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:23.955111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:23.955111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:23.955111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:23.955111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:23.955111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:23.955111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:23.955111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:23.955111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:23.956004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:23.956004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:23.956004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:23.990023 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:23.990509 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:23.990701 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:23.990931 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:37:23.991328 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:23.991538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:23.992135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:23.992135 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:23.992135 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:23.992135 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:23.992135 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:23.992135 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:23.992135 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:23.992135 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:23.993361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:23.993361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:37:23.993361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:23.993361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:23.993361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:23.993361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:23.993361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:23.993361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:23.993361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:23.993361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:23.993361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:23.993361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:37:23.993361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:23.993361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:37:23.993361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:37:23.994185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:23.994185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:37:23.994185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:23.994185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:37:23.994185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:37:23.994185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:23.994185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:37:23.994185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:23.994185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:37:23.994669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:23.994669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:23.994669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:23.994669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:23.994669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:23.994669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:37:23.994669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:37:23.994669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:37:23.994669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:23.994669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:23.994669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:23.994669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:23.995292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:23.995292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:23.995292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:23.995292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:23.995292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:23.995292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:23.995292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:23.995292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:23.995292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:23.995292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:23.995292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:23.995292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:23.995292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:23.995292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:23.995958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:23.995958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:23.995958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:23.995958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:23.996208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:23.996208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:23.996208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:23.996208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:23.996208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:23.996208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:23.996208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:23.996208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:23.996208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:23.996208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:23.996208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:23.996208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:23.996208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:23.996208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:23.996208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:23.996208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:23.997075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:37:23.997075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:37:23.997075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:37:23.997075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:23.997075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:23.997075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:23.997075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:23.997075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:23.997075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:23.997075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:23.997075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:23.997075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:23.997075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:23.997075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:23.997075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:23.997075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:23.997075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:23.997075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:23.997952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:23.997952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:23.997952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:23.997952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:23.997952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:23.997952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:23.997952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:23.997952 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:23.997952 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:23.997952 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:23.998454 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:23.998454 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:23.998743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:23.999289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:23.999289 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:23.999289 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:23.999289 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:23.999289 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:23.999289 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:23.999289 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:23.999289 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:24.000454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:24.000454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:37:24.000454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:24.000454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:24.000454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:24.000454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:24.000454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:24.000454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:24.000454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:24.000454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:24.000454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:24.000454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:37:24.000454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:24.000454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:37:24.000454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:37:24.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:24.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:37:24.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:24.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:37:24.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:37:24.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:24.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:37:24.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:24.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:37:24.001693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:24.001693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:24.001693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:24.001693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:24.001693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:24.001693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:37:24.001693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:37:24.001693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:24.001693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:24.001693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:24.001693 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:24.001992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:24.001992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:24.001992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:24.001992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:24.001992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:24.001992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:24.001992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:24.001992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:24.001992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:24.001992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:24.001992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:24.001992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:24.001992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:24.001992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:24.002349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:24.002349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:24.002349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:24.002349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:24.002790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:24.002790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:24.002790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:24.002790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:24.002790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:24.002790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:24.002790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:24.002790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:24.002790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:24.002790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:24.002790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:24.002790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:24.002790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:24.002790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:24.002790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:24.002790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:24.002790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:37:24.003277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:37:24.003277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:24.003277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:24.003277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:24.003277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:24.003277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:24.003277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:24.003277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:24.003277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:24.003277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:24.003277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:24.003277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:24.003277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:24.003277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:24.003277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:24.003277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:24.003277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:24.003277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:24.003277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:24.003957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:24.003957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:24.003957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:24.003957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:24.005311 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:24.005542 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:24.005654 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:24.005840 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:24.006157 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:24.006413 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:24.006618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:24.007296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:24.007296 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:24.007296 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:24.007296 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:24.007296 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:24.007296 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:24.007296 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:24.007296 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:24.008167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:24.008167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:24.008167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:24.008167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:24.008167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:24.008167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:37:24.008167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:24.008167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:24.008167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:24.008167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:24.008167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:24.008167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:24.008552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:24.008552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:24.008552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:24.008552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:24.008552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:24.008552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:24.008552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:24.008552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:24.008552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:24.008552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:24.008552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:24.008552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:24.008552 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:24.008986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:24.008986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:24.009210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:24.009210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:24.009210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:24.009210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:24.009210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:24.009210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:24.009210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:24.009210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:24.009210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:24.009210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:24.009210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:24.009210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:24.009210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:24.009210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:24.009210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:24.009210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:24.009650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:37:24.009650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:24.009650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:24.009650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:24.009650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:24.009650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:24.009650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:24.009650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:24.009650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:24.009650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:24.009650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:24.009650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:24.009650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:24.009650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:24.009650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:24.009650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:24.009650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:24.009650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:24.009650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:24.010177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:24.010177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:24.010177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:28.719797 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:28.720450 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:28.720637 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:28.721886 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:28.755271 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:28.755743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:28.756560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:28.756560 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:28.756560 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:28.756560 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:28.756560 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:28.756560 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:28.756560 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:28.756560 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:28.757330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:28.757330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:28.757330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:28.757330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:28.757330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:28.757330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:37:28.757330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:28.757330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:28.757330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:28.757330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:28.757330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:28.757330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:28.757764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:28.757764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:28.757764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:28.757764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:28.757764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:28.757764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:28.757764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:28.757764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:28.757764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:28.757764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:28.757764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:28.757764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:28.757764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:28.758242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:28.758242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:28.758242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:28.758242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:28.758242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:28.758242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:28.758242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:28.758242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:28.758242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:28.758242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:28.758242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:28.758242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:28.758242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:28.758242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:28.758242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:28.758742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:28.758742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:28.758742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:28.758742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:37:28.758742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:28.758742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:28.758742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:28.758742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:28.758742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:28.758742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:28.758742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:28.758742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:28.758742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:28.758742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:28.758742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:28.758742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:28.758742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:28.758742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:28.758742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:28.759306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:28.759306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:28.759306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:28.759306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:28.759306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:28.759306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:28.948258 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:28.948697 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:28.948936 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:28.949881 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:28.950120 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:28.950281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:28.951007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:28.951007 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:28.951007 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:28.951007 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:28.951007 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:28.951007 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:28.951007 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:28.951007 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:28.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:28.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:28.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:28.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:28.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:28.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:37:28.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:28.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:28.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:28.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:28.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:28.952412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:28.953108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:28.953108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:28.953108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:28.953108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:28.953108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:28.953108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:28.953108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:28.953108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:28.953108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:28.953108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:28.953108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:28.953108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:28.953108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:28.953651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:28.953651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:28.954061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:28.954061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:28.954061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:28.954061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:28.954061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:28.954061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:28.954061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:28.954061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:28.954061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:28.954061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:28.954061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:28.954061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:28.954061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:28.954061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:28.954061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:28.954061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:28.954830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:37:28.954830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:28.954830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:28.954830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:28.954830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:28.954830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:28.954830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:28.954830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:28.954830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:28.954830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:28.954830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:28.954830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:28.954830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:28.954830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:28.954830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:28.954830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:28.954830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:28.954830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:28.954830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:28.955670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:28.955670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:28.955670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:28.990763 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:28.991117 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:28.991318 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:28.991496 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:37:28.991929 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:28.992343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:28.993048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:28.993048 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:28.993048 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:28.993048 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:28.993048 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:28.993048 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:28.993048 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:28.993048 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:28.994219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:28.994219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:37:28.994219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:28.994219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:28.994219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:28.994219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:28.994219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:28.994219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:28.994219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:28.994219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:28.994219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:28.994219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:37:28.994219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:28.994219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:37:28.994219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:37:28.995288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:28.995288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:37:28.995288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:28.995288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:37:28.995288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:37:28.995288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:28.995288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:37:28.995288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:28.995288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:37:28.995866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:28.995866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:28.995866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:28.995866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:28.995866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:28.995866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:37:28.995866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:37:28.995866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:37:28.995866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:28.995866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:28.995866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:28.995866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:28.996559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:28.996559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:28.996559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:28.996559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:28.996559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:28.996559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:28.996559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:28.996559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:28.996559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:28.996559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:28.996559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:28.996559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:28.996559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:28.996559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:28.997548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:28.997548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:28.997548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:28.997548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:28.997548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:28.997548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:28.997548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:28.997548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:28.997548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:28.997548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:28.997548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:28.997548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:28.997548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:28.997548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:28.998523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:28.998523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:28.998523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:28.998523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:28.998523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:28.998523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:28.998523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:37:28.998523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:37:28.998523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:37:28.998523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:28.998523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:28.998523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:28.998523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:28.998523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:28.998523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:28.998523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:28.998523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:28.998523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:28.998523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:28.999733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:28.999733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:28.999733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:28.999733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:28.999733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:28.999733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:28.999733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:28.999733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:28.999733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:28.999733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:28.999733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:28.999733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:28.999733 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:28.999733 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:29.000227 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:29.000227 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:29.000227 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:29.000227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:29.000394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:29.000394 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:29.000394 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:29.000394 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:29.000394 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:29.000394 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:29.000394 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:29.000394 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:29.001166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:29.001166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:37:29.001166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:29.001166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:29.001166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:29.001166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:29.001166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:29.001166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:29.001166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:29.001166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:29.001166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:29.001166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:37:29.001166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:29.001166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:37:29.001166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:37:29.001651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:29.001651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:37:29.001651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:29.001651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:37:29.001651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:37:29.001651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:29.001651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:37:29.001651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:29.001651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:37:29.002066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:29.002066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:29.002066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:29.002066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:29.002066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:29.002066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:37:29.002066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:37:29.002066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:29.002066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:29.002066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:29.002066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:29.002486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:29.002486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:29.002486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:29.002486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:29.002486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:29.002486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:29.002486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:29.002486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:29.002486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:29.002486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:29.002486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:29.002486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:29.002486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:29.002486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:29.002971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:29.002971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:29.002971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:29.002971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:29.003198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:29.003198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:29.003198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:29.003198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:29.003198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:29.003198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:29.003198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:29.003198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:29.003198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:29.003198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:29.003198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:29.003198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:29.003198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:29.003198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:29.003198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:29.003198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:29.003198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:37:29.003772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:37:29.003772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:29.003772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:29.003772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:29.003772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:29.003772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:29.003772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:29.003772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:29.003772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:29.003772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:29.003772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:29.003772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:29.003772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:29.003772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:29.003772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:29.003772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:29.003772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:29.003772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:29.003772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:29.004223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:29.004223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:29.004223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:29.004223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:29.005630 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:29.005872 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:29.005987 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:29.006141 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:29.006841 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:29.007064 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:29.007206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:29.007669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:29.007669 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:29.007669 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:29.007669 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:29.007669 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:29.007669 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:29.007669 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:29.007669 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:29.008514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:29.008514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:29.008514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:29.008514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:29.008514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:29.008514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:37:29.008514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:29.008514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:29.008514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:29.008514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:29.008514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:29.008514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:29.008888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:29.008888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:29.008888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:29.008888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:29.008888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:29.008888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:29.008888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:29.008888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:29.008888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:29.008888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:29.008888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:29.008888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:29.008888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:29.009149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:29.009149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:29.009521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:29.009521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:29.009521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:29.009521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:29.009521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:29.009521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:29.009521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:29.009521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:29.009521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:29.009521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:29.009521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:29.009521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:29.009521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:29.009521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:29.009521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:29.009521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:29.009902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:37:29.009902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:29.009902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:29.009902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:29.009902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:29.009902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:29.009902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:29.009902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:29.009902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:29.009902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:29.009902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:29.009902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:29.009902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:29.009902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:29.009902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:29.009902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:29.009902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:29.009902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:29.009902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:29.010251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:29.010251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:29.010251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:33.720485 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:33.721234 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:33.721288 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:33.722576 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:33.754475 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:33.755005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:33.755858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:33.755858 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:33.755858 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:33.755858 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:33.755858 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:33.755858 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:33.755858 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:33.755858 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:33.756576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:33.756576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:33.756576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:33.756576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:33.756576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:33.756576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:37:33.756576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:33.756576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:33.756576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:33.756576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:33.756576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:33.756576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:33.757606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:33.757606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:33.757606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:33.757606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:33.757606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:33.757606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:33.757606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:33.757606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:33.757606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:33.757606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:33.757606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:33.757606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:33.757606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:33.758620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:33.758620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:33.758620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:33.758620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:33.758620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:33.758620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:33.758620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:33.758620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:33.758620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:33.758620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:33.758620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:33.758620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:33.758620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:33.758620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:33.758620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:33.759862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:33.759862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:33.759862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:33.759862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:37:33.759862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:33.759862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:33.759862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:33.759862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:33.759862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:33.759862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:33.759862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:33.759862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:33.759862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:33.759862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:33.759862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:33.759862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:33.759862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:33.759862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:33.759862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:33.761266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:33.761266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:33.761266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:33.761266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:33.761266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:33.761266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:33.949130 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:33.949510 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:33.949765 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:33.950826 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:33.951268 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:33.951570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:33.952802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:33.952802 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:33.952802 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:33.952802 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:33.952802 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:33.952802 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:33.952802 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:33.952802 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:33.954247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:33.954247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:33.954247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:33.954247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:33.954247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:33.954247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:37:33.954247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:33.954247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:33.954247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:33.954247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:33.954247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:33.954247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:33.954875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:33.954875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:33.954875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:33.954875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:33.954875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:33.954875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:33.954875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:33.954875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:33.954875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:33.954875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:33.954875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:33.954875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:33.954875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:33.955483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:33.955483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:33.956093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:33.956093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:33.956093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:33.956093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:33.956093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:33.956093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:33.956093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:33.956093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:33.956093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:33.956093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:33.956093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:33.956093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:33.956093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:33.956093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:33.956093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:33.956093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:33.956867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:37:33.956867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:33.956867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:33.956867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:33.956867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:33.956867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:33.956867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:33.956867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:33.956867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:33.956867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:33.956867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:33.956867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:33.956867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:33.956867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:33.956867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:33.956867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:33.956867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:33.956867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:33.956867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:33.957742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:33.957742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:33.957742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:33.991534 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:33.991909 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:33.992089 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:33.992271 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:37:33.993006 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:33.993249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:33.993997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:33.993997 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:33.993997 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:33.993997 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:33.993997 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:33.993997 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:33.993997 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:33.993997 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:33.995201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:33.995201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:37:33.995201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:33.995201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:33.995201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:33.995201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:33.995201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:33.995201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:33.995201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:33.995201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:33.995201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:33.995201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:37:33.995201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:33.995201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:37:33.995201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:37:33.996162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:33.996162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:37:33.996162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:33.996162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:37:33.996162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:37:33.996162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:33.996162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:37:33.996162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:33.996162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:37:33.996602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:33.996602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:33.996602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:33.996602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:33.996602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:33.996602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:37:33.996602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:37:33.996602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:37:33.996602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:33.996602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:33.996602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:33.996602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:33.997213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:33.997213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:33.997213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:33.997213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:33.997213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:33.997213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:33.997213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:33.997213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:33.997213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:33.997213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:33.997213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:33.997213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:33.997213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:33.997213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:33.997877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:33.997877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:33.997877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:33.997877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:33.998228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:33.998228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:33.998228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:33.998228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:33.998228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:33.998228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:33.998228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:33.998228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:33.998228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:33.998228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:33.998228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:33.998228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:33.998228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:33.998228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:33.998228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:33.998228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:33.998944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:37:33.998944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:37:33.998944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:37:33.998944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:33.998944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:33.998944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:33.998944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:33.998944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:33.998944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:33.998944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:33.998944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:33.998944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:33.998944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:33.998944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:33.998944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:33.998944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:33.998944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:33.998944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:33.999818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:33.999818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:33.999818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:33.999818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:33.999818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:33.999818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:33.999818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:33.999818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:33.999818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:33.999818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:34.000298 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:34.000298 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:34.000298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:34.000298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:34.000298 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:34.000298 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:34.000298 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:34.000298 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:34.000298 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:34.000298 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:34.000298 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:34.000828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:34.000828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:37:34.000828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:34.000828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:34.000828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:34.000828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:34.000828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:34.000828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:34.000828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:34.000828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:34.000828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:34.000828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:37:34.000828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:34.000828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:37:34.000828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:37:34.001452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:34.001452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:37:34.001452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:34.001452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:37:34.001452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:37:34.001452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:34.001452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:37:34.001452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:34.001452 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:37:34.001916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:34.001916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:34.001916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:34.001916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:34.001916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:34.001916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:37:34.001916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:37:34.001916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:34.001916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:34.001916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:34.001916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:34.002456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:34.002456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:34.002456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:34.002456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:34.002456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:34.002456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:34.002456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:34.002456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:34.002456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:34.002456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:34.002456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:34.002456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:34.002456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:34.002456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:34.002974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:34.002974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:34.002974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:34.002974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:34.002974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:34.002974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:34.002974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:34.002974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:34.002974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:34.002974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:34.002974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:34.002974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:34.002974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:34.002974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:34.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:34.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:34.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:34.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:34.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:34.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:34.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:37:34.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:37:34.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:34.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:34.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:34.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:34.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:34.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:34.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:34.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:34.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:34.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:34.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:34.003968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:34.003968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:34.003968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:34.003968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:34.003968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:34.003968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:34.003968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:34.003968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:34.003968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:34.003968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:34.003968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:34.006239 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:34.006495 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:34.006601 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:34.006773 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:34.007096 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:34.007318 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:34.007465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:34.007966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:34.007966 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:34.007966 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:34.007966 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:34.007966 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:34.007966 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:34.007966 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:34.007966 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:34.008770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:34.008770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:34.008770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:34.008770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:34.008770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:34.008770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:37:34.008770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:34.008770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:34.008770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:34.008770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:34.008770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:34.008770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:34.009214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:34.009214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:34.009214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:34.009214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:34.009214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:34.009214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:34.009214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:34.009214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:34.009214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:34.009214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:34.009214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:34.009214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:34.009214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:34.009593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:34.009593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:34.009741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:34.009741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:34.009741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:34.009741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:34.009741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:34.009741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:34.009741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:34.009741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:34.009741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:34.009741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:34.009741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:34.009741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:34.009741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:34.009741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:34.009741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:34.009741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:34.010253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:37:34.010253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:34.010253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:34.010253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:34.010253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:34.010253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:34.010253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:34.010253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:34.010253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:34.010253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:34.010253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:34.010253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:34.010253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:34.010253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:34.010253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:34.010253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:34.010253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:34.010253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:34.010253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:34.010840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:34.010840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:34.010840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:38.721139 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:38.721848 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:38.721971 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:38.723194 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:38.757720 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:38.758213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:38.759186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:38.759186 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:38.759186 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:38.759186 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:38.759186 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:38.759186 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:38.759186 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:38.759186 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:38.760073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:38.760073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:38.760073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:38.760073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:38.760073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:38.760073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:37:38.760073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:38.760073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:38.760073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:38.760073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:38.760073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:38.760073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:38.760497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:38.760497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:38.760497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:38.760497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:38.760497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:38.760497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:38.760497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:38.760497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:38.760497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:38.760497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:38.760497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:38.760497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:38.760497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:38.760924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:38.760924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:38.760924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:38.760924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:38.760924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:38.760924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:38.760924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:38.760924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:38.760924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:38.760924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:38.760924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:38.760924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:38.760924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:38.760924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:38.760924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:38.761381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:38.761381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:38.761381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:38.761381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:37:38.761381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:38.761381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:38.761381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:38.761381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:38.761381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:38.761381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:38.761381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:38.761381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:38.761381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:38.761381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:38.761381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:38.761381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:38.761381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:38.761381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:38.761381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:38.761951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:38.761951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:38.761951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:38.761951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:38.761951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:38.761951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:38.949937 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:38.950446 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:38.950556 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:38.951483 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:38.951910 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:38.952277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:38.953154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:38.953154 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:38.953154 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:38.953154 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:38.953154 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:38.953154 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:38.953154 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:38.953154 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:38.954181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:38.954181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:38.954181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:38.954181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:38.954181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:38.954181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:37:38.954181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:38.954181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:38.954181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:38.954181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:38.954181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:38.954181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:38.954774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:38.954774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:38.954774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:38.954774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:38.954774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:38.954774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:38.954774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:38.954774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:38.954774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:38.954774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:38.954774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:38.954774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:38.954774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:38.955347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:38.955347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:38.955347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:38.955347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:38.955347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:38.955347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:38.955347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:38.955347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:38.955347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:38.955347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:38.955347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:38.955347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:38.955347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:38.955347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:38.955347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:38.956023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:38.956023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:38.956023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:38.956023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:37:38.956023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:38.956023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:38.956023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:38.956023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:38.956023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:38.956023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:38.956023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:38.956023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:38.956023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:38.956023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:38.956023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:38.956023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:38.956023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:38.956023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:38.956023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:38.956881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:38.956881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:38.956881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:38.956881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:38.956881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:38.956881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:38.991968 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:38.992302 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:38.992387 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:38.992534 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:37:38.993096 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:38.993444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:38.994183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:38.994183 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:38.994183 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:38.994183 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:38.994183 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:38.994183 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:38.994183 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:38.994183 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:38.995001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:38.995001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:37:38.995001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:38.995001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:38.995001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:38.995001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:38.995001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:38.995001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:38.995001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:38.995001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:38.995001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:38.995001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:37:38.995001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:38.995001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:37:38.995001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:37:38.995482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:38.995482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:37:38.995482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:38.995482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:37:38.995482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:37:38.995482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:38.995482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:37:38.995482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:38.995482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:37:38.995798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:38.995798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:38.995798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:38.995798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:38.995798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:38.995798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:37:38.995798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:37:38.995798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:37:38.995798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:38.995798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:38.995798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:38.995798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:38.996140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:38.996140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:38.996140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:38.996140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:38.996140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:38.996140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:38.996140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:38.996140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:38.996140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:38.996140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:38.996140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:38.996140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:38.996140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:38.996140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:38.996513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:38.996513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:38.996513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:38.996513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:38.996850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:38.996850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:38.996850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:38.996850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:38.996850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:38.996850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:38.996850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:38.996850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:38.996850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:38.996850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:38.996850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:38.996850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:38.996850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:38.996850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:38.996850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:38.996850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:38.997448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:37:38.997448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:37:38.997448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:37:38.997448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:38.997448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:38.997448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:38.997448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:38.997448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:38.997448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:38.997448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:38.997448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:38.997448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:38.997448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:38.997448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:38.997448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:38.997448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:38.997448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:38.997448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:38.998003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:38.998003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:38.998003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:38.998003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:38.998003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:38.998003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:38.998003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:38.999009 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:38.999300 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:38.999397 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:38.999528 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:39.000033 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:39.000538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:39.001296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:39.001296 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:39.001296 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:39.001296 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:39.001296 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:39.001296 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:39.001296 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:39.001296 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:39.002026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:39.002026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:37:39.002026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:39.002026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:39.002026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:39.002026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:39.002026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:39.002026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:39.002026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:39.002026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:39.002026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:39.002026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:37:39.002026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:39.002026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:37:39.002026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:37:39.002402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:39.002402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:37:39.002402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:39.002402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:37:39.002402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:37:39.002402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:39.002402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:37:39.002402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:39.002402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:37:39.002895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:39.002895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:39.002895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:39.002895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:39.002895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:39.002895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:37:39.002895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:37:39.002895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:39.002895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:39.002895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:39.002895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:39.003247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:39.003247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:39.003247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:39.003247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:39.003247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:39.003247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:39.003247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:39.003247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:39.003247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:39.003247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:39.003247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:39.003247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:39.003247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:39.003247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:39.003639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:39.003639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:39.003639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:39.003639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:39.003935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:39.003935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:39.003935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:39.003935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:39.003935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:39.003935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:39.003935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:39.003935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:39.003935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:39.003935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:39.003935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:39.003935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:39.003935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:39.003935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:39.003935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:39.003935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:39.003935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:37:39.004658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:37:39.004658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:39.004658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:39.004658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:39.004658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:39.004658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:39.004658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:39.004658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:39.004658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:39.004658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:39.004658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:39.004658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:39.004658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:39.004658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:39.004658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:39.004658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:39.004658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:39.004658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:39.004658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:39.005748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:39.005748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:39.005748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:39.005748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:39.006559 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:39.006843 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:39.006942 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:39.007128 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:39.007452 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:39.007723 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:39.007898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:39.008465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:39.008465 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:39.008465 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:39.008465 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:39.008465 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:39.008465 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:39.008465 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:39.008465 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:39.009311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:39.009311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:39.009311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:39.009311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:39.009311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:39.009311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:37:39.009311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:39.009311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:39.009311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:39.009311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:39.009311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:39.009311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:39.009737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:39.009737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:39.009737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:39.009737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:39.009737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:39.009737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:39.009737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:39.009737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:39.009737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:39.009737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:39.009737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:39.009737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:39.009737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:39.010116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:39.010116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:39.010297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:39.010297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:39.010297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:39.010297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:39.010297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:39.010297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:39.010297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:39.010297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:39.010297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:39.010297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:39.010297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:39.010297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:39.010297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:39.010297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:39.010297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:39.010297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:39.010894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:37:39.010894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:39.010894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:39.010894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:39.010894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:39.010894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:39.010894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:39.010894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:39.010894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:39.010894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:39.010894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:39.010894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:39.010894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:39.010894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:39.010894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:39.010894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:39.010894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:39.010894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:39.010894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:39.011405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:39.011405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:39.011405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:43.721724 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:43.722614 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:43.722614 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:43.723895 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:43.755810 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:43.756216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:43.757027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:43.757027 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:43.757027 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:43.757027 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:43.757027 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:43.757027 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:43.757027 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:43.757027 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:43.757841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:43.757841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:43.757841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:43.757841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:43.757841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:43.757841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:37:43.757841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:43.757841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:43.757841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:43.757841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:43.757841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:43.757841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:43.758542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:43.758542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:43.758542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:43.758542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:43.758542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:43.758542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:43.758542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:43.758542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:43.758542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:43.758542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:43.758542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:43.758542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:43.758542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:43.759222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:43.759222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:43.759222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:43.759222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:43.759222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:43.759222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:43.759222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:43.759222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:43.759222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:43.759222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:43.759222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:43.759222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:43.759222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:43.759222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:43.759222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:43.759658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:43.759658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:43.759658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:43.759658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:37:43.759658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:43.759658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:43.759658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:43.759658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:43.759658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:43.759658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:43.759658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:43.759658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:43.759658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:43.759658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:43.759658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:43.759658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:43.759658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:43.759658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:43.759658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:43.760222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:43.760222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:43.760222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:43.760222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:43.760222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:43.760222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:43.950587 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:43.951282 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:43.951498 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:43.952798 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:43.953145 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:43.953382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:43.954308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:43.954308 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:43.954308 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:43.954308 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:43.954308 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:43.954308 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:43.954308 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:43.954308 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:43.955856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:43.955856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:43.955856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:43.955856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:43.955856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:43.955856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:37:43.955856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:43.955856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:43.955856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:43.955856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:43.955856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:43.955856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:43.956835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:43.956835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:43.956835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:43.956835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:43.956835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:43.956835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:43.956835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:43.956835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:43.956835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:43.956835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:43.956835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:43.956835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:43.956835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:43.957764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:43.957764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:43.957764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:43.957764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:43.957764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:43.957764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:43.957764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:43.957764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:43.957764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:43.957764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:43.957764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:43.957764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:43.957764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:43.957764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:43.957764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:43.958725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:43.958725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:43.958725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:43.958725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:37:43.958725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:43.958725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:43.958725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:43.958725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:43.958725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:43.958725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:43.958725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:43.958725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:43.958725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:43.958725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:43.958725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:43.958725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:43.958725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:43.958725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:43.958725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:43.959712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:43.959712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:43.959712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:43.959712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:43.959712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:43.959712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:43.992541 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:43.992889 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:43.993065 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:43.993242 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:37:43.993830 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:43.994132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:43.994959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:43.994959 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:43.994959 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:43.994959 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:43.994959 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:43.994959 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:43.994959 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:43.994959 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:43.996137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:43.996137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:37:43.996137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:43.996137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:43.996137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:43.996137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:43.996137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:43.996137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:43.996137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:43.996137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:43.996137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:43.996137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:37:43.996137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:43.996137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:37:43.996137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:37:43.996975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:43.996975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:37:43.996975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:43.996975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:37:43.996975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:37:43.996975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:43.996975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:37:43.996975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:43.996975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:37:43.997576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:43.997576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:43.997576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:43.997576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:43.997576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:43.997576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:37:43.997576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:37:43.997576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:37:43.997576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:43.997576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:43.997576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:43.997576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:43.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:43.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:43.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:43.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:43.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:43.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:43.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:43.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:43.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:43.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:43.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:43.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:43.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:43.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:43.998804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:43.998804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:43.998804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:43.998804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:43.999139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:43.999139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:43.999139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:43.999139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:43.999139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:43.999139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:43.999139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:43.999139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:43.999139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:43.999139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:43.999139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:43.999139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:43.999139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:43.999139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:43.999139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:43.999139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:43.999913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:37:43.999913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:37:43.999913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:37:43.999913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:43.999913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:43.999913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:43.999913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:43.999913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:43.999913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:43.999913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:43.999913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:43.999913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:43.999913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:43.999913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:43.999913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:43.999913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:43.999913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:43.999913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:44.000802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:44.000802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:44.000802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:44.000802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:44.000802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:44.000802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:44.000802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:44.000802 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:44.000802 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:44.000802 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:44.001315 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:44.001315 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:44.001315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:44.001503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:44.001503 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:44.001503 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:44.001503 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:44.001503 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:44.001503 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:44.001503 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:44.001503 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:44.002146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:44.002146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:37:44.002146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:44.002146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:44.002146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:44.002146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:44.002146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:44.002146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:44.002146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:44.002146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:44.002146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:44.002146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:37:44.002146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:44.002146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:37:44.002146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:37:44.002567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:44.002567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:37:44.002567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:44.002567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:37:44.002567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:37:44.002567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:44.002567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:37:44.002567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:44.002567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:37:44.003042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:44.003042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:44.003042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:44.003042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:44.003042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:44.003042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:37:44.003042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:37:44.003042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:44.003042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:44.003042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:44.003042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:44.003374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:44.003374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:44.003374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:44.003374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:44.003374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:44.003374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:44.003374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:44.003374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:44.003374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:44.003374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:44.003374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:44.003374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:44.003374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:44.003374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:44.003774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:44.003774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:44.003774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:44.003774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:44.004098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:44.004098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:44.004098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:44.004098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:44.004098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:44.004098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:44.004098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:44.004098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:44.004098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:44.004098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:44.004098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:44.004098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:44.004098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:44.004098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:44.004098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:44.004098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:44.004098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:37:44.004700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:37:44.004700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:44.004700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:44.004700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:44.004700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:44.004700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:44.004700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:44.004700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:44.004700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:44.004700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:44.004700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:44.004700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:44.004700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:44.004700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:44.004700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:44.004700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:44.004700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:44.004700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:44.004700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:44.005210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:44.005210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:44.005210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:44.005210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:44.006908 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:44.007358 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:44.007561 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:44.007934 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:44.008365 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:44.008740 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:44.008956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:44.009798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:44.009798 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:44.009798 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:44.009798 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:44.009798 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:44.009798 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:44.009798 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:44.009798 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:44.011344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:44.011344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:44.011344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:44.011344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:44.011344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:44.011344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:37:44.011344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:44.011344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:44.011344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:44.011344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:44.011344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:44.011344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:44.011739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:44.011739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:44.011739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:44.011739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:44.011739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:44.011739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:44.011739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:44.011739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:44.011739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:44.011739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:44.011739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:44.011739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:44.011739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:44.012103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:44.012103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:44.013357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:44.013357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:44.013357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:44.013357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:44.013357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:44.013357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:44.013357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:44.013357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:44.013357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:44.013357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:44.013357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:44.013357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:44.013357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:44.013357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:44.013357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:44.013357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:44.013848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:37:44.013848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:44.013848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:44.013848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:44.013848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:44.013848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:44.013848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:44.013848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:44.013848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:44.013848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:44.013848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:44.013848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:44.013848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:44.013848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:44.013848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:44.013848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:44.013848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:44.013848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:44.013848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:44.014375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:44.014375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:44.014375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:48.722307 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:48.722912 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:48.723102 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:48.724314 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:48.758803 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:48.759206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:48.759840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:48.759840 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:48.759840 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:48.759840 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:48.759840 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:48.759840 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:48.759840 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:48.759840 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:48.760609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:48.760609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:48.760609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:48.760609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:48.760609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:48.760609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:37:48.760609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:48.760609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:48.760609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:48.760609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:48.760609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:48.760609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:48.761263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:48.761263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:48.761263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:48.761263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:48.761263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:48.761263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:48.761263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:48.761263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:48.761263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:48.761263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:48.761263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:48.761263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:48.761263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:48.762144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:48.762144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:48.762144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:48.762144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:48.762144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:48.762144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:48.762144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:48.762144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:48.762144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:48.762144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:48.762144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:48.762144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:48.762144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:48.762144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:48.762144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:48.762943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:48.762943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:48.762943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:48.762943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:37:48.762943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:48.762943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:48.762943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:48.762943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:48.762943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:48.762943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:48.762943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:48.762943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:48.762943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:48.762943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:48.762943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:48.762943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:48.762943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:48.762943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:48.762943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:48.763899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:48.763899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:48.763899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:48.763899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:48.763899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:48.763899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:48.951576 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:48.952282 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:48.952447 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:48.953700 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:48.954019 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:48.954242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:48.955132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:48.955132 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:48.955132 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:48.955132 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:48.955132 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:48.955132 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:48.955132 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:48.955132 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:48.956700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:48.956700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:48.956700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:48.956700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:48.956700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:48.956700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:37:48.956700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:48.956700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:48.956700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:48.956700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:48.956700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:48.956700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:48.957430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:48.957430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:48.957430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:48.957430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:48.957430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:48.957430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:48.957430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:48.957430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:48.957430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:48.957430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:48.957430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:48.957430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:48.957430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:48.958205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:48.958205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:48.958422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:48.958422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:48.958422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:48.958422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:48.958422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:48.958422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:48.958422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:48.958422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:48.958422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:48.958422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:48.958422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:48.958422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:48.958422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:48.958422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:48.958422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:48.958422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:48.959206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:37:48.959206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:48.959206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:48.959206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:48.959206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:48.959206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:48.959206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:48.959206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:48.959206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:48.959206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:48.959206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:48.959206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:48.959206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:48.959206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:48.959206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:48.959206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:48.959206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:48.959206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:48.959206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:48.960083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:48.960083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:48.960083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:48.993002 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:48.993335 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:48.993494 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:48.993575 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:37:48.994156 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:48.994473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:48.995195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:48.995195 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:48.995195 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:48.995195 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:48.995195 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:48.995195 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:48.995195 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:48.995195 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:48.995974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:48.995974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:37:48.995974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:48.995974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:48.995974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:48.995974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:48.995974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:48.995974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:48.995974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:48.995974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:48.995974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:48.995974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:37:48.995974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:48.995974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:37:48.995974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:37:48.996734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:48.996734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:37:48.996734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:48.996734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:37:48.996734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:37:48.996734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:48.996734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:37:48.996734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:48.996734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:37:48.997306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:48.997306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:48.997306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:48.997306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:48.997306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:48.997306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:37:48.997306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:37:48.997306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:37:48.997306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:48.997306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:48.997306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:48.997306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:48.998115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:48.998115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:48.998115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:48.998115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:48.998115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:48.998115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:48.998115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:48.998115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:48.998115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:48.998115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:48.998115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:48.998115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:48.998115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:48.998115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:48.998948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:48.998948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:48.998948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:48.998948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:48.998948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:48.998948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:48.998948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:48.998948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:48.998948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:48.998948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:48.998948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:48.998948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:48.998948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:48.998948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:48.999834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:48.999834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:48.999834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:48.999834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:48.999834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:48.999834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:48.999834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:37:48.999834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:37:48.999834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:37:48.999834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:48.999834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:48.999834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:48.999834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:48.999834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:48.999834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:48.999834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:48.999834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:48.999834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:48.999834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:49.001024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:49.001024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:49.001024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:49.001024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:49.001024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:49.001024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:49.001024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:49.001024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:49.001024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:49.001024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:49.001024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:49.001024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:49.001024 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:49.001024 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:49.002094 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:49.002094 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:49.002094 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:49.002094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:49.002094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:49.002094 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:49.002094 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:49.002094 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:49.002094 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:49.002094 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:49.002094 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:49.002094 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:49.002611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:49.002611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:37:49.002611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:49.002611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:49.002611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:49.002611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:49.002611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:49.002611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:49.002611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:49.002611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:49.002611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:49.002611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:37:49.002611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:49.002611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:37:49.002611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:37:49.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:49.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:37:49.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:49.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:37:49.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:37:49.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:49.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:37:49.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:49.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:37:49.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:49.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:49.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:49.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:49.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:49.003779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:37:49.003779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:37:49.003779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:49.003779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:49.003779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:49.003779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:49.003779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:49.003779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:49.003779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:49.003779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:49.003779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:49.003779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:49.003779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:49.004219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:49.004219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:49.004219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:49.004219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:49.004219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:49.004219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:49.004219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:49.004219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:49.004219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:49.004219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:49.004219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:49.004219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:49.004697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:49.004697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:49.004697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:49.004697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:49.004697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:49.004697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:49.004697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:49.004697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:49.004697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:49.004697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:49.004697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:49.004697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:49.004697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:49.004697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:49.004697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:49.004697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:37:49.004697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:37:49.004697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:49.004697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:49.005985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:49.005985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:49.005985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:49.005985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:49.005985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:49.005985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:49.005985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:49.005985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:49.005985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:49.005985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:49.005985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:49.005985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:49.005985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:49.005985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:49.005985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:49.005985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:49.005985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:49.005985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:49.005985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:49.005985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:49.007310 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:49.007550 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:49.007665 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:49.007847 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:49.008097 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:49.008252 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:49.008356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:49.008776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:49.008776 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:49.008776 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:49.008776 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:49.008776 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:49.008776 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:49.008776 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:49.008776 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:49.009658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:49.009658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:49.009658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:49.009658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:49.009658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:49.009658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:37:49.009658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:49.009658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:49.009658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:49.009658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:49.009658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:49.009658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:49.010032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:49.010032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:49.010032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:49.010032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:49.010032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:49.010032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:49.010032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:49.010032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:49.010032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:49.010032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:49.010032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:49.010032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:49.010032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:49.010723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:49.010723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:49.010723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:49.010723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:49.010723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:49.010723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:49.010723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:49.010723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:49.010723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:49.010723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:49.010723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:49.010723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:49.010723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:49.010723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:49.010723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:49.011194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:49.011194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:49.011194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:49.011194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:37:49.011194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:49.011194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:49.011194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:49.011194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:49.011194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:49.011194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:49.011194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:49.011194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:49.011194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:49.011194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:49.011194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:49.011194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:49.011194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:49.011194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:49.011194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:49.011641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:49.011641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:49.011641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:49.011641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:49.011641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:49.011641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:53.722910 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:53.723770 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:53.723770 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:53.724463 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:53.757915 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:53.758936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:53.759249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:53.759249 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:53.759249 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:53.759249 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:53.759249 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:53.759249 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:53.759249 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:53.759249 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:53.760265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:53.760265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:53.760265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:53.760265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:53.760265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:53.760265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:37:53.760265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:53.760265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:53.760265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:53.760265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:53.760265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:53.760265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:53.760861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:53.760861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:53.760861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:53.760861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:53.760861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:53.760861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:53.760861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:53.760861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:53.760861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:53.760861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:53.760861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:53.760861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:53.760861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:53.761436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:53.761436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:53.761436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:53.761436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:53.761436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:53.761436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:53.761436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:53.761436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:53.761436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:53.761436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:53.761436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:53.761436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:53.761436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:53.761436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:53.761436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:53.762082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:53.762082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:53.762082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:53.762082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:37:53.762082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:53.762082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:53.762082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:53.762082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:53.762082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:53.762082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:53.762082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:53.762082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:53.762082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:53.762082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:53.762082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:53.762082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:53.762082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:53.762082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:53.762082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:53.762922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:53.762922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:53.762922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:53.762922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:53.762922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:53.762922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:53.952254 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:53.952815 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:53.952909 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:53.953894 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:53.954208 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:53.954431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:53.955248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:53.955248 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:53.955248 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:53.955248 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:53.955248 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:53.955248 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:53.955248 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:53.955248 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:53.956351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:53.956351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:53.956351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:53.956351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:53.956351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:53.956351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:37:53.956351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:53.956351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:53.956351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:53.956351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:53.956351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:53.956351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:53.956953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:53.956953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:53.956953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:53.956953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:53.956953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:53.956953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:53.956953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:53.956953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:53.956953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:53.956953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:53.956953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:53.956953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:53.956953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:53.957526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:53.957526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:53.957526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:53.957526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:53.957526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:53.957526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:53.957526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:53.957526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:53.957526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:53.957526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:53.957526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:53.957526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:53.957526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:53.957526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:53.957526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:53.958208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:53.958208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:53.958208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:53.958208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:37:53.958208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:53.958208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:53.958208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:53.958208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:53.958208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:53.958208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:53.958208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:53.958208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:53.958208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:53.958208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:53.958208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:53.958208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:53.958208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:53.958208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:53.958208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:53.959020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:53.959020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:53.959020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:53.959020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:53.959020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:53.959020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:53.993533 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:53.994113 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:53.994396 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:53.994648 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:37:53.995377 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:53.995718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:53.996624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:53.996624 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:53.996624 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:53.996624 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:53.996624 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:53.996624 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:53.996624 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:53.996624 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:53.998255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:53.998255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:37:53.998255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:53.998255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:53.998255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:53.998255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:53.998255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:53.998255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:53.998255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:53.998255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:53.998255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:53.998255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:37:53.998255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:53.998255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:37:53.998255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:37:53.998996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:53.998996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:37:53.998996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:53.998996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:37:53.998996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:37:53.998996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:53.998996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:37:53.998996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:53.998996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:37:53.999706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:53.999706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:53.999706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:53.999706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:53.999706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:53.999706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:37:53.999706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:37:53.999706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:37:53.999706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:53.999706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:53.999706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:53.999706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:54.000363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:54.000363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:54.000363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:54.000363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:54.000363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:54.000363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:54.000363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:54.000363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:54.000363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:54.000363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:54.000363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:54.000363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:54.000363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:54.000363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:54.001074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:54.001074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:54.001074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:54.001074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:54.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:54.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:54.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:54.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:54.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:54.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:54.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:54.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:54.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:54.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:54.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:54.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:54.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:54.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:54.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:54.001385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:54.002105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:37:54.002105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:37:54.002105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:37:54.002105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:54.002105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:54.002105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:54.002105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:54.002105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:54.002105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:54.002105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:54.002105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:54.002105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:54.002105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:54.002105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:54.002105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:54.002105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:54.002105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:54.002105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:54.002944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:54.002944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:54.002944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:54.002944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:54.002944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:54.002944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:54.002944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:54.002944 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:54.002944 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:54.002944 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:54.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:54.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:54.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:54.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:54.003397 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:54.003397 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:54.003397 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:54.003397 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:54.003397 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:54.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:54.003397 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:54.004239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:54.004239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:37:54.004239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:54.004239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:54.004239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:54.004239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:54.004239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:54.004239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:54.004239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:54.004239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:54.004239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:54.004239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:37:54.004239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:54.004239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:37:54.004239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:37:54.004964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:54.004964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:37:54.004964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:54.004964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:37:54.004964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:37:54.004964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:54.004964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:37:54.004964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:54.004964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:37:54.005221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:54.005221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:54.005221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:54.005221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:54.005221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:54.005221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:37:54.005221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:37:54.005221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:54.005221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:54.005221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:54.005221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:54.005529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:54.005529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:54.005529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:54.005529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:54.005529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:54.005529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:54.005529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:54.005529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:54.005529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:54.005529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:54.005529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:54.005529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:54.005529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:54.005529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:54.005910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:54.005910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:54.005910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:54.005910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:54.006321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:54.006321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:54.006321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:54.006321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:54.006321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:54.006321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:54.006321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:54.006321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:54.006321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:54.006321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:54.006321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:54.006321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:54.006321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:54.006321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:54.006321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:54.006321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:54.006321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:37:54.006820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:37:54.006820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:54.006820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:54.006820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:54.006820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:54.006820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:54.006820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:54.006820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:54.006820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:54.006820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:54.006820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:54.006820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:54.006820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:54.006820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:54.006820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:54.006820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:54.006820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:54.006820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:54.006820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:54.007343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:54.007343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:54.007343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:54.007343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:54.007613 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:54.007888 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:54.008003 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:54.008208 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:54.008521 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:54.008771 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:54.008916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:54.009424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:54.009424 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:54.009424 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:54.009424 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:54.009424 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:54.009424 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:54.009424 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:54.009424 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:54.010249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:54.010249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:54.010249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:54.010249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:54.010249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:54.010249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:37:54.010249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:54.010249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:54.010249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:54.010249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:54.010249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:54.010249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:54.010591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:54.010591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:54.010591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:54.010591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:54.010591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:54.010591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:54.010591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:54.010591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:54.010591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:54.010591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:54.010591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:54.010591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:54.010591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:54.010978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:54.010978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:54.011273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:54.011273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:54.011273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:54.011273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:54.011273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:54.011273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:54.011273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:54.011273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:54.011273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:54.011273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:54.011273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:54.011273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:54.011273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:54.011273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:54.011273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:54.011273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:54.011748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:37:54.011748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:54.011748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:54.011748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:54.011748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:54.011748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:54.011748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:54.011748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:54.011748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:54.011748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:54.011748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:54.011748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:54.011748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:54.011748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:54.011748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:54.011748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:54.011748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:54.011748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:54.011748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:54.012340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:54.012340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:54.012340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:58.723470 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:58.724090 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:58.724292 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:58.725530 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:58.759152 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:58.759398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:58.760013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:58.760013 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:58.760013 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:58.760013 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:58.760013 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:58.760013 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:58.760013 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:58.760013 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:58.760773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:58.760773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:58.760773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:58.760773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:58.760773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:58.760773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:37:58.760773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:58.760773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:58.760773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:58.760773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:58.760773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:58.760773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:58.761937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:58.761937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:58.761937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:58.761937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:58.761937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:58.761937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:58.761937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:58.761937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:58.761937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:58.761937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:58.761937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:58.761937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:58.761937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:58.762899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:58.762899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:58.762899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:58.762899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:58.762899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:58.762899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:58.762899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:58.762899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:58.762899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:58.762899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:58.762899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:58.762899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:58.762899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:58.762899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:58.762899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:58.763556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:58.763556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:58.763556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:58.763556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:37:58.763556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:58.763556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:58.763556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:58.763556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:58.763556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:58.763556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:58.763556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:58.763556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:58.763556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:58.763556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:58.763556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:58.763556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:58.763556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:58.763556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:58.763556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:58.764369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:58.764369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:58.764369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:58.764369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:58.764369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:58.764369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:58.952904 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:58.953441 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:58.953646 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:58.954907 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:58.955231 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:58.955495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:58.956386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:58.956386 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:58.956386 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:58.956386 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:58.956386 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:58.956386 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:58.956386 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:58.956386 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:58.958002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:58.958002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:58.958002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:58.958002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:58.958002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:58.958002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:37:58.958002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:58.958002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:58.958002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:58.958002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:58.958002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:58.958002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:58.958497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:58.958497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:58.958497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:58.958497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:58.958497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:58.958497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:58.958497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:58.958497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:58.958497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:58.958497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:58.958497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:58.958497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:58.958497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:58.959070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:58.959070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:58.959775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:58.959775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:58.959775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:58.959775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:58.959775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:58.959775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:58.959775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:58.959775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:58.959775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:58.959775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:58.959775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:58.959775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:58.959775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:58.959775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:58.959775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:58.959775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:58.961254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:37:58.961254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:58.961254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:58.961254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:58.961254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:58.961254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:58.961254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:58.961254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:58.961254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:58.961254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:58.961254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:58.961254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:58.961254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:58.961254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:58.961254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:58.961254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:58.961254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:58.961254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:58.961254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:58.962507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:58.962507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:58.962507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:58.994311 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:58.995036 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:58.995036 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:58.995833 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:37:58.995833 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:58.996376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:58.997383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:58.997383 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:58.997383 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:58.997383 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:58.997383 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:58.997383 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:58.997383 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:58.997383 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:58.998583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:58.998583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:37:58.998583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:58.998583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:58.998583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:58.998583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:58.998583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:58.998583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:58.998583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:58.998583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:58.998583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:58.998583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:37:58.998583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:58.998583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:37:58.998583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:37:58.999246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:58.999246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:37:58.999246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:58.999246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:37:58.999246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:37:58.999246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:58.999246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:37:58.999246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:58.999246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:37:58.999926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:58.999926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:58.999926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:58.999926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:58.999926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:58.999926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:37:58.999926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:37:58.999926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:37:58.999926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:58.999926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:58.999926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:58.999926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:59.000352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:59.000352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:59.000352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:59.000352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:59.000352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:59.000352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:59.000352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:59.000352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:59.000352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:59.000352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:59.000352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:59.000352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:59.000352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:59.000352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:59.001093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:59.001093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:59.001093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:59.001093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:59.001534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:59.001534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:59.001534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:59.001534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:59.001534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:59.001534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:59.001534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:59.001534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:59.001534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:59.001534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:59.001534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:59.001534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:59.001534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:59.001534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:59.001534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:59.001534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:59.002047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:37:59.002047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:37:59.002047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:37:59.002047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:59.002047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:59.002047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:59.002047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:59.002047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:59.002047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:59.002047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:59.002047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:59.002047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:59.002047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:59.002047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:59.002047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:59.002047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:59.002047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:59.002047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:59.002599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:59.002599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:59.002599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:59.002599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:59.002599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:59.002599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:59.002599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:59.002599 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:59.002599 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:59.002599 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:59.003971 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:59.003971 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:59.003971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:59.004105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:59.004105 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:59.004105 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:59.004105 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:59.004105 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:59.004105 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:59.004105 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:59.004105 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:59.005193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:59.005193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:37:59.005193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:59.005193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:59.005193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:59.005193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:59.005193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:59.005193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:59.005193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:59.005193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:37:59.005193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:59.005193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:37:59.005193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:59.005193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:37:59.005193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:37:59.005596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:59.005596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:37:59.005596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:59.005596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:37:59.005596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:37:59.005596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:59.005596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:37:59.005596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:59.005596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:37:59.006618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:59.006618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:59.006618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:59.006618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:59.006618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:59.006618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:37:59.006618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:37:59.006618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:59.006618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:59.006618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:59.006618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:59.006955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:59.006955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:59.006955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:59.006955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:59.006955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:59.006955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:59.006955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:59.006955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:59.006955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:59.006955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:59.006955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:59.006955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:59.006955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:59.006955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:59.007271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:59.007271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:59.007271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:59.007271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:59.008143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:59.008143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:59.008143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:59.008143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:59.008143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:59.008143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:59.008143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:59.008143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:59.008143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:59.008143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:59.008143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:59.008143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:59.008143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:59.008143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:59.008143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:59.008143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:59.008143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:37:59.008624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:37:59.008624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:37:59.008624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:37:59.008624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:59.008624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:59.008624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:59.008624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:59.008624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:59.008624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:59.008624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:59.008624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:59.008624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:59.008624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:59.008624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:59.008624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:59.008624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:59.008624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:59.008624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:59.008624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:59.009080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:59.009080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:59.009080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:59.009080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:37:59.009080 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:37:59.009080 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:37:59.009080 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:37:59.009080 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:37:59.009371 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:59.009462 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:37:59.009616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:59.010059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:37:59.010059 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:59.010059 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:37:59.010059 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:37:59.010059 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:59.010059 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:37:59.010059 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:59.010059 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:37:59.010976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:59.010976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:37:59.010976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:59.010976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:37:59.010976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:59.010976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:37:59.010976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:59.010976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:59.010976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:59.010976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:59.010976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:59.010976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:59.011276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:37:59.011276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:59.011276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:37:59.011276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:59.011276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:37:59.011276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:37:59.011276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:59.011276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:37:59.011276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:59.011276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:37:59.011276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:37:59.011276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:59.011276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:37:59.011611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:59.011611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:37:59.011935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:59.011935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:37:59.011935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:37:59.011935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:37:59.011935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:37:59.011935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:37:59.011935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:59.011935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:37:59.011935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:37:59.011935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:59.011935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:37:59.011935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:37:59.011935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:37:59.011935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:37:59.011935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:37:59.011935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:37:59.012329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:37:59.012329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:37:59.012329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:59.012329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:59.012329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:37:59.012329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:37:59.012329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:37:59.012329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:37:59.012329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:37:59.012329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:37:59.012329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:37:59.012329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:37:59.012329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:37:59.012329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:37:59.012329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:37:59.012329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:37:59.012329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:37:59.012329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:37:59.012329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:37:59.012774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:37:59.012774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:37:59.012774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:03.724072 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:03.726873 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:03.726873 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:03.726873 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:03.758097 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:03.758423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:03.759101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:03.759101 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:03.759101 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:03.759101 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:03.759101 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:03.759101 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:03.759101 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:03.759101 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:03.759899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:03.759899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:03.759899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:03.759899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:03.759899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:03.759899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:38:03.759899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:03.759899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:03.759899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:03.759899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:03.759899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:03.759899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:03.760761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:03.760761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:03.760761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:03.760761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:03.760761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:03.760761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:03.760761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:03.760761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:03.760761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:03.760761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:03.760761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:03.760761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:03.760761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:03.761623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:03.761623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:03.761623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:03.761623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:03.761623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:03.761623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:03.761623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:03.761623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:03.761623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:03.761623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:03.761623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:03.761623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:03.761623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:03.761623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:03.761623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:03.762449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:03.762449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:03.762449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:03.762449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:38:03.762449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:03.762449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:03.762449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:03.762449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:03.762449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:03.762449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:03.762449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:03.762449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:03.762449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:03.762449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:03.762449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:03.762449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:03.762449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:03.762449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:03.762449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:03.764107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:03.764107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:03.764107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:03.764107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:03.764107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:03.764107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:03.953668 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:03.954220 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:03.954413 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:03.955644 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:03.956220 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:03.956457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:03.957417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:03.957417 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:03.957417 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:03.957417 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:03.957417 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:03.957417 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:03.957417 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:03.957417 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:03.958971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:03.958971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:03.958971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:03.958971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:03.958971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:03.958971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:38:03.958971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:03.958971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:03.958971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:03.958971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:03.958971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:03.958971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:03.959562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:03.959562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:03.959562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:03.959562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:03.959562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:03.959562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:03.959562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:03.959562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:03.959562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:03.959562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:03.959562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:03.959562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:03.959562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:03.960168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:03.960168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:03.960618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:03.960618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:03.960618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:03.960618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:03.960618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:03.960618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:03.960618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:03.960618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:03.960618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:03.960618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:03.960618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:03.960618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:03.960618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:03.960618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:03.960618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:03.960618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:03.961152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:38:03.961152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:03.961152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:03.961152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:03.961152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:03.961152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:03.961152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:03.961152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:03.961152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:03.961152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:03.961152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:03.961152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:03.961152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:03.961152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:03.961152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:03.961152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:03.961152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:03.961152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:03.961152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:03.961814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:03.961814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:03.961814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:03.994887 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:03.995325 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:03.995475 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:03.995662 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:38:03.996212 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:03.996539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:03.997456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:03.997456 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:03.997456 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:03.997456 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:03.997456 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:03.997456 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:03.997456 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:03.997456 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:03.998722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:03.998722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:38:03.998722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:03.998722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:03.998722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:03.998722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:03.998722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:03.998722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:03.998722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:03.998722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:03.998722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:03.998722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:38:03.998722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:03.998722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:38:03.998722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:38:03.999363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:03.999363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:38:03.999363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:03.999363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:38:03.999363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:38:03.999363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:03.999363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:38:03.999363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:03.999363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:38:04.000029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:04.000029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:04.000029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:04.000029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:04.000029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:04.000029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:38:04.000029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:38:04.000029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:38:04.000029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:04.000029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:04.000029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:04.000029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:04.000561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:04.000561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:04.000561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:04.000561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:04.000561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:04.000561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:04.000561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:04.000561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:04.000561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:04.000561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:04.000561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:04.000561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:04.000561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:04.000561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:04.001133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:04.001133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:04.001133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:04.001133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:04.001668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:04.001668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:04.001668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:04.001668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:04.001668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:04.001668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:04.001668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:04.001668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:04.001668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:04.001668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:04.001668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:04.001668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:04.001668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:04.001668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:04.001668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:04.001668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:04.002166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:38:04.002166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:38:04.002166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:38:04.002166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:04.002166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:04.002166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:04.002166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:04.002166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:04.002166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:04.002166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:04.002166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:04.002166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:04.002166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:04.002166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:04.002166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:04.002166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:04.002166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:04.002166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:04.002636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:04.002636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:04.002636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:04.002636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:04.002636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:04.002636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:04.002636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:04.002636 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:04.002636 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:04.002636 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:04.002909 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:04.003131 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:04.003482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:04.004167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:04.004167 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:04.004167 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:04.004167 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:04.004167 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:04.004167 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:04.004167 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:04.004167 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:04.004910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:04.004910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:38:04.004910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:04.004910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:04.004910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:04.004910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:04.004910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:04.004910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:04.004910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:04.004910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:04.004910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:04.004910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:38:04.004910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:04.004910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:38:04.004910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:38:04.005283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:04.005283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:38:04.005283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:04.005283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:38:04.005283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:38:04.005283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:04.005283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:38:04.005283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:04.005283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:38:04.005797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:04.005797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:04.005797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:04.005797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:04.005797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:04.005797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:38:04.005797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:38:04.005797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:04.005797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:04.005797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:04.005797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:04.006101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:04.006101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:04.006101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:04.006101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:04.006101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:04.006101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:04.006101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:04.006101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:04.006101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:04.006101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:04.006101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:04.006101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:04.006101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:04.006101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:04.006448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:04.006448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:04.006448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:04.006448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:04.006826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:04.006826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:04.006826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:04.006826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:04.006826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:04.006826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:04.006826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:04.006826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:04.006826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:04.006826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:04.006826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:04.006826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:04.006826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:04.006826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:04.006826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:04.006826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:04.006826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:38:04.007233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:38:04.007233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:04.007233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:04.007233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:04.007233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:04.007233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:04.007233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:04.007233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:04.007233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:04.007233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:04.007233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:04.007233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:04.007233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:04.007233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:04.007233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:04.007233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:04.007233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:04.007233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:04.007233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:04.007733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:04.007733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:04.007733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:04.007733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:04.008983 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:04.009157 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:04.009223 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:04.009416 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:04.009890 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:04.010288 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:04.010439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:04.011184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:04.011184 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:04.011184 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:04.011184 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:04.011184 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:04.011184 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:04.011184 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:04.011184 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:04.012006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:04.012006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:04.012006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:04.012006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:04.012006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:04.012006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:38:04.012006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:04.012006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:04.012006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:04.012006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:04.012006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:04.012006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:04.012325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:04.012325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:04.012325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:04.012325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:04.012325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:04.012325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:04.012325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:04.012325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:04.012325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:04.012325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:04.012325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:04.012325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:04.012325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:04.012668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:04.012668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:04.012980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:04.012980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:04.012980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:04.012980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:04.012980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:04.012980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:04.012980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:04.012980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:04.012980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:04.012980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:04.012980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:04.012980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:04.012980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:04.012980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:04.012980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:04.012980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:04.013474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:38:04.013474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:04.013474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:04.013474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:04.013474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:04.013474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:04.013474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:04.013474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:04.013474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:04.013474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:04.013474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:04.013474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:04.013474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:04.013474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:04.013474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:04.013474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:04.013474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:04.013474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:04.013474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:04.013968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:04.013968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:04.013968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:08.724729 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:08.725583 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:08.725583 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:08.726819 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:08.748864 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=238 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:38:08.758275 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:08.758731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:08.759663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:08.759663 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:08.759663 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:08.759663 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:08.759663 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:08.759663 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:08.759663 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:08.759663 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:08.760364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:08.760364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:08.760364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:08.760364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:08.760364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:08.760364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:38:08.760364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:08.760364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:08.760364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:08.760364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:08.760364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:08.760364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:08.761033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:08.761033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:08.761033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:08.761033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:08.761033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:08.761033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:08.761033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:08.761033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:08.761033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:08.761033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:08.761033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:08.761033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:08.761033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:08.761708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:08.761708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:08.761708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:08.761708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:08.761708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:08.761708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:08.761708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:08.761708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:08.761708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:08.761708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:08.761708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:08.761708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:08.761708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:08.761708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:08.761708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:08.762467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:08.762467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:08.762467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:08.762467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:38:08.762467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:08.762467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:08.762467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:08.762467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:08.762467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:08.762467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:08.762467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:08.762467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:08.762467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:08.762467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:08.762467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:08.762467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:08.762467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:08.762467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:08.762467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:08.763434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:08.763434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:08.763434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:08.763434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:08.763434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:08.763434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:08.954277 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:08.955763 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:08.955763 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:08.956845 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:08.957189 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:08.957422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:08.958517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:08.958517 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:08.958517 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:08.958517 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:08.958517 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:08.958517 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:08.958517 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:08.958517 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:08.959696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:08.959696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:08.959696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:08.959696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:08.959696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:08.959696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:38:08.959696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:08.959696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:08.959696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:08.959696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:08.959696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:08.959696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:08.960537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:08.960537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:08.960537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:08.960537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:08.960537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:08.960537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:08.960537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:08.960537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:08.960537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:08.960537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:08.960537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:08.960537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:08.960537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:08.961297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:08.961297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:08.961297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:08.961297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:08.961297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:08.961297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:08.961297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:08.961297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:08.961297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:08.961297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:08.961297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:08.961297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:08.961297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:08.961297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:08.961297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:08.961979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:08.961979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:08.961979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:08.961979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:38:08.961979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:08.961979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:08.961979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:08.961979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:08.961979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:08.961979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:08.961979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:08.961979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:08.961979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:08.961979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:08.961979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:08.961979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:08.961979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:08.961979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:08.961979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:08.965077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:08.965077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:08.965077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:08.965077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:08.965077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:08.965077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:08.995489 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:08.996048 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:08.996208 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:08.996394 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:38:08.997077 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:08.997491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:08.998411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:08.998411 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:08.998411 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:08.998411 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:08.998411 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:08.998411 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:08.998411 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:08.998411 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:08.999698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:08.999698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:38:08.999698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:08.999698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:08.999698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:08.999698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:08.999698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:08.999698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:08.999698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:08.999698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:08.999698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:08.999698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:38:08.999698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:08.999698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:38:08.999698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:38:09.000498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:09.000498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:38:09.000498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:09.000498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:38:09.000498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:38:09.000498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:09.000498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:38:09.000498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:09.000498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:38:09.001079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:09.001079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:09.001079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:09.001079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:09.001079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:09.001079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:38:09.001079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:38:09.001079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:38:09.001079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:09.001079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:09.001079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:09.001079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:09.001695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:09.001695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:09.001695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:09.001695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:09.001695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:09.001695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:09.001695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:09.001695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:09.001695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:09.001695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:09.001695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:09.001695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:09.001695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:09.001695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:09.002299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:09.002299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:09.002299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:09.002299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:09.002299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:09.002299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:09.002299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:09.002299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:09.002299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:09.002299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:09.002299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:09.002299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:09.002299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:09.002299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:09.002939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:09.002939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:09.002939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:09.002939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:09.002939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:09.002939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:09.002939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:38:09.002939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:38:09.002939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:38:09.002939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:09.002939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:09.002939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:09.002939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:09.002939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:09.002939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:09.002939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:09.002939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:09.002939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:09.002939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:09.003834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:09.003834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:09.003834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:09.003834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:09.003834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:09.003834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:09.003834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:09.003834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:09.003834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:09.003834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:09.003834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:09.003834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:09.003834 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:09.003834 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:09.004470 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:09.004470 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:09.004470 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:09.004470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:09.005078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:09.005078 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:09.005078 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:09.005078 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:09.005078 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:09.005078 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:09.005078 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:09.005078 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:09.006341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:09.006341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:38:09.006341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:09.006341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:09.006341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:09.006341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:09.006341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:09.006341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:09.006341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:09.006341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:09.006341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:09.006341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:38:09.006341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:09.006341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:38:09.006341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:38:09.006754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:09.006754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:38:09.006754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:09.006754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:38:09.006754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:38:09.006754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:09.006754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:38:09.006754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:09.006754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:38:09.007889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:09.007889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:09.007889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:09.007889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:09.007889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:09.007889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:38:09.007889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:38:09.007889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:09.007889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:09.007889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:09.007889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:09.008234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:09.008234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:09.008234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:09.008234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:09.008234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:09.008234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:09.008234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:09.008234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:09.008234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:09.008234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:09.008234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:09.008234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:09.008234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:09.008234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:09.008588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:09.008588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:09.008588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:09.008588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:09.009502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:09.009502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:09.009502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:09.009502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:09.009502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:09.009502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:09.009502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:09.009502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:09.009502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:09.009502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:09.009502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:09.009502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:09.009502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:09.009502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:09.009502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:09.009502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:09.009502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:38:09.009993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:38:09.009993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:09.009993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:09.009993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:09.009993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:09.009993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:09.009993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:09.009993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:09.009993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:09.009993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:09.009993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:09.009993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:09.009993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:09.009993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:09.009993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:09.009993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:09.009993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:09.009993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:09.009993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:09.010491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:09.010491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:09.010491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:09.010491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:09.010491 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:09.010646 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:09.010749 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:09.010895 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:09.011409 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:09.011849 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:09.012141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:09.012924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:09.012924 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:09.012924 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:09.012924 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:09.012924 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:09.012924 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:09.012924 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:09.012924 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:09.013761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:09.013761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:09.013761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:09.013761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:09.013761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:09.013761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:38:09.013761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:09.013761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:09.013761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:09.013761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:09.013761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:09.013761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:09.014105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:09.014105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:09.014105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:09.014105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:09.014105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:09.014105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:09.014105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:09.014105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:09.014105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:09.014105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:09.014105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:09.014105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:09.014105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:09.014448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:09.014448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:09.014804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:09.014804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:09.014804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:09.014804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:09.014804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:09.014804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:09.014804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:09.014804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:09.014804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:09.014804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:09.014804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:09.014804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:09.014804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:09.014804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:09.014804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:09.014804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:09.015235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:38:09.015235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:09.015235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:09.015235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:09.015235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:09.015235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:09.015235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:09.015235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:09.015235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:09.015235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:09.015235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:09.015235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:09.015235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:09.015235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:09.015235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:09.015235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:09.015235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:09.015235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:09.015235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:09.015767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:09.015767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:09.015767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:13.725392 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:13.726323 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:13.726323 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:13.727093 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:13.755790 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:13.756064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:13.756787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:13.756787 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:13.756787 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:13.756787 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:13.756787 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:13.756787 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:13.756787 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:13.756787 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:13.757760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:13.757760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:13.757760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:13.757760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:13.757760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:13.757760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:38:13.757760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:13.757760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:13.757760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:13.757760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:13.757760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:13.757760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:13.758335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:13.758335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:13.758335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:13.758335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:13.758335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:13.758335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:13.758335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:13.758335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:13.758335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:13.758335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:13.758335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:13.758335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:13.758335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:13.758964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:13.758964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:13.758964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:13.758964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:13.758964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:13.758964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:13.758964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:13.758964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:13.758964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:13.758964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:13.758964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:13.758964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:13.758964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:13.758964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:13.758964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:13.759584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:13.759584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:13.759584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:13.759584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:38:13.759584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:13.759584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:13.759584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:13.759584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:13.759584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:13.759584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:13.759584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:13.759584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:13.759584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:13.759584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:13.759584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:13.759584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:13.759584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:13.759584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:13.759584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:13.760351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:13.760351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:13.760351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:13.760351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:13.760351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:13.760351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:13.955351 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:13.956001 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:13.956249 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:13.957230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:13.957596 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:13.957896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:13.958726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:13.958726 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:13.958726 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:13.958726 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:13.958726 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:13.958726 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:13.958726 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:13.958726 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:13.960442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:13.960442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:13.960442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:13.960442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:13.960442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:13.960442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:38:13.960442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:13.960442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:13.960442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:13.960442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:13.960442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:13.960442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:13.962878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:13.962878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:13.962878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:13.962878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:13.962878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:13.962878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:13.962878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:13.962878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:13.962878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:13.962878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:13.962878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:13.962878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:13.962878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:13.963628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:13.963628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:13.963628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:13.963628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:13.963628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:13.963628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:13.963628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:13.963628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:13.963628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:13.963628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:13.963628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:13.963628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:13.963628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:13.963628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:13.963628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:13.964229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:13.964229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:13.964229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:13.964229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:38:13.964229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:13.964229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:13.964229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:13.964229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:13.964229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:13.964229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:13.964229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:13.964229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:13.964229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:13.964229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:13.964229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:13.964229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:13.964229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:13.964229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:13.964229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:13.964964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:13.964964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:13.964964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:13.964964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:13.964964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:13.964964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:13.996079 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:13.996411 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:13.996646 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:13.997044 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:38:13.997428 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:13.997673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:13.998482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:13.998482 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:13.998482 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:13.998482 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:13.998482 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:13.998482 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:13.998482 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:13.998482 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:13.999996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:13.999996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:38:13.999996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:13.999996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:13.999996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:13.999996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:13.999996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:13.999996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:13.999996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:13.999996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:13.999996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:13.999996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:38:13.999996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:13.999996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:38:13.999996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:38:14.000849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:14.000849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:38:14.000849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:14.000849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:38:14.000849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:38:14.000849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:14.000849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:38:14.000849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:14.000849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:38:14.001819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:14.001819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:14.001819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:14.001819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:14.001819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:14.001819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:38:14.001819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:38:14.001819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:38:14.001819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:14.001819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:14.001819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:14.001819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:14.002312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:14.002312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:14.002312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:14.002312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:14.002312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:14.002312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:14.002312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:14.002312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:14.002312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:14.002312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:14.002312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:14.002312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:14.002312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:14.002312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:14.002765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:14.002765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:14.002765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:14.002765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:14.003857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:14.003857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:14.003857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:14.003857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:14.003857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:14.003857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:14.003857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:14.003857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:14.003857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:14.003857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:14.003857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:14.003857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:14.003857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:14.003857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:14.003857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:14.003857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:14.004301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:38:14.004301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:38:14.004301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:38:14.004301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:14.004301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:14.004301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:14.004301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:14.004301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:14.004301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:14.004301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:14.004301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:14.004301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:14.004301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:14.004301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:14.004301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:14.004301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:14.004301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:14.004301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:14.004819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:14.004819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:14.004819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:14.004819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:14.004819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:14.004819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:14.004819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:14.004819 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:14.004819 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:14.004819 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:14.005125 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:14.005230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:14.005576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:14.006278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:14.006278 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:14.006278 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:14.006278 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:14.006278 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:14.006278 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:14.006278 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:14.006278 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:14.007162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:14.007162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:38:14.007162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:14.007162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:14.007162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:14.007162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:14.007162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:14.007162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:14.007162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:14.007162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:14.007162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:14.007162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:38:14.007162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:14.007162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:38:14.007162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:38:14.007581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:14.007581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:38:14.007581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:14.007581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:38:14.007581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:38:14.007581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:14.007581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:38:14.007581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:14.007581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:38:14.008311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:14.008311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:14.008311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:14.008311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:14.008311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:14.008311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:38:14.008311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:38:14.008311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:14.008311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:14.008311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:14.008311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:14.008635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:14.008635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:14.008635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:14.008635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:14.008635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:14.008635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:14.008635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:14.008635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:14.008635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:14.008635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:14.008635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:14.008635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:14.008635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:14.008635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:14.009026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:14.009026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:14.009026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:14.009026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:14.009617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:14.009617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:14.009617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:14.009617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:14.009617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:14.009617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:14.009617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:14.009617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:14.009617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:14.009617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:14.009617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:14.009617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:14.009617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:14.009617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:14.009617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:14.009617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:14.009617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:38:14.010156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:38:14.010156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:14.010156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:14.010156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:14.010156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:14.010156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:14.010156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:14.010156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:14.010156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:14.010156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:14.010156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:14.010156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:14.010156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:14.010156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:14.010156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:14.010156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:14.010156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:14.010156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:14.010156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:14.010745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:14.010745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:14.010745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:14.010745 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:14.010745 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:14.011114 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:14.011269 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:14.011468 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:14.011887 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:14.012229 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:14.012448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:14.013147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:14.013147 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:14.013147 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:14.013147 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:14.013147 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:14.013147 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:14.013147 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:14.013147 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:14.014152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:14.014152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:14.014152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:14.014152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:14.014152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:14.014152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:38:14.014152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:14.014152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:14.014152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:14.014152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:14.014152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:14.014152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:14.014513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:14.014513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:14.014513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:14.014513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:14.014513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:14.014513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:14.014513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:14.014513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:14.014513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:14.014513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:14.014513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:14.014513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:14.014513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:14.014948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:14.014948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:14.015497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:14.015497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:14.015497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:14.015497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:14.015497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:14.015497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:14.015497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:14.015497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:14.015497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:14.015497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:14.015497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:14.015497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:14.015497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:14.015497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:14.015497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:14.015497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:14.015978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:38:14.015978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:14.015978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:14.015978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:14.015978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:14.015978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:14.015978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:14.015978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:14.015978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:14.015978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:14.015978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:14.015978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:14.015978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:14.015978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:14.015978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:14.015978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:14.015978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:14.015978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:14.015978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:14.016484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:14.016484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:14.016484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:18.726122 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:18.727095 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:18.727095 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:18.728560 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:18.760053 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:18.760498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:18.761338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:18.761338 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:18.761338 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:18.761338 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:18.761338 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:18.761338 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:18.761338 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:18.761338 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:18.762092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:18.762092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:18.762092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:18.762092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:18.762092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:18.762092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:38:18.762092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:18.762092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:18.762092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:18.762092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:18.762092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:18.762092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:18.762851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:18.762851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:18.762851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:18.762851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:18.762851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:18.762851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:18.762851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:18.762851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:18.762851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:18.762851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:18.762851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:18.762851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:18.762851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:18.763543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:18.763543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:18.763543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:18.763543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:18.763543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:18.763543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:18.763543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:18.763543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:18.763543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:18.763543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:18.763543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:18.763543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:18.763543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:18.763543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:18.763543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:18.764415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:18.764415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:18.764415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:18.764415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:38:18.764415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:18.764415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:18.764415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:18.764415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:18.764415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:18.764415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:18.764415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:18.764415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:18.764415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:18.764415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:18.764415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:18.764415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:18.764415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:18.764415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:18.764415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:18.765459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:18.765459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:18.765459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:18.765459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:18.765459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:18.765459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:18.956158 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:18.956806 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:18.957028 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:18.958190 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:18.958554 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:18.958867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:18.959761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:18.959761 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:18.959761 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:18.959761 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:18.959761 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:18.959761 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:18.959761 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:18.959761 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:18.961540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:18.961540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:18.961540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:18.961540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:18.961540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:18.961540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:38:18.961540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:18.961540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:18.961540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:18.961540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:18.961540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:18.961540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:18.962138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:18.962138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:18.962138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:18.962138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:18.962138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:18.962138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:18.962138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:18.962138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:18.962138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:18.962138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:18.962138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:18.962138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:18.962138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:18.962710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:18.962710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:18.963458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:18.963458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:18.963458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:18.963458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:18.963458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:18.963458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:18.963458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:18.963458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:18.963458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:18.963458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:18.963458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:18.963458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:18.963458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:18.963458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:18.963458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:18.963458 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:18.964178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:38:18.964178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:18.964178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:18.964178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:18.964178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:18.964178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:18.964178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:18.964178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:18.964178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:18.964178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:18.964178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:18.964178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:18.964178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:18.964178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:18.964178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:18.964178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:18.964178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:18.964178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:18.964178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:18.965007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:18.965007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:18.965007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:18.996468 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:18.996850 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:18.997043 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:18.997272 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:38:18.997826 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:18.998187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:18.999032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:18.999032 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:18.999032 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:18.999032 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:18.999032 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:18.999032 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:18.999032 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:18.999032 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:19.000379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:19.000379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:38:19.000379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:19.000379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:19.000379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:19.000379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:19.000379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:19.000379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:19.000379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:19.000379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:19.000379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:19.000379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:38:19.000379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:19.000379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:38:19.000379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:38:19.001164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:19.001164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:38:19.001164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:19.001164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:38:19.001164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:38:19.001164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:19.001164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:38:19.001164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:19.001164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:38:19.002046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:19.002046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:19.002046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:19.002046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:19.002046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:19.002046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:38:19.002046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:38:19.002046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:38:19.002046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:19.002046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:19.002046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:19.002046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:19.002615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:19.002615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:19.002615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:19.002615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:19.002615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:19.002615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:19.002615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:19.002615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:19.002615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:19.002615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:19.002615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:19.002615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:19.002615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:19.002615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:19.003229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:19.003229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:19.003229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:19.003229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:19.003883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:19.003883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:19.003883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:19.003883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:19.003883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:19.003883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:19.003883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:19.003883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:19.003883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:19.003883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:19.003883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:19.003883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:19.003883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:19.003883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:19.003883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:19.003883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:19.004622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:38:19.004622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:38:19.004622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:38:19.004622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:19.004622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:19.004622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:19.004622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:19.004622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:19.004622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:19.004622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:19.004622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:19.004622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:19.004622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:19.004622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:19.004622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:19.004622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:19.004622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:19.004622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:19.005471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:19.005471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:19.005471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:19.005471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:19.005471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:19.005471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:19.005471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:19.005471 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:19.005471 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:19.005471 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:19.005976 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:19.005976 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:19.006218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:19.006974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:19.006974 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:19.006974 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:19.006974 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:19.006974 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:19.006974 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:19.006974 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:19.006974 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:19.007761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:19.007761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:38:19.007761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:19.007761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:19.007761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:19.007761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:19.007761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:19.007761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:19.007761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:19.007761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:19.007761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:19.007761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:38:19.007761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:19.007761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:38:19.007761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:38:19.008191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:19.008191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:38:19.008191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:19.008191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:38:19.008191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:38:19.008191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:19.008191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:38:19.008191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:19.008191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:38:19.008664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:19.008664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:19.008664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:19.008664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:19.008664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:19.008664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:38:19.008664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:38:19.008664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:19.008664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:19.008664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:19.008664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:19.009017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:19.009017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:19.009017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:19.009017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:19.009017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:19.009017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:19.009017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:19.009017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:19.009017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:19.009017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:19.009017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:19.009017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:19.009017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:19.009017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:19.009509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:19.009509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:19.009509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:19.009509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:19.009796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:19.009796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:19.009796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:19.009796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:19.009796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:19.009796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:19.009796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:19.009796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:19.009796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:19.009796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:19.009796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:19.009796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:19.009796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:19.009796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:19.009796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:19.009796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:19.009796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:38:19.010394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:38:19.010394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:19.010394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:19.010394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:19.010394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:19.010394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:19.010394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:19.010394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:19.010394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:19.010394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:19.010394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:19.010394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:19.010394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:19.010394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:19.010394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:19.010394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:19.010394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:19.010394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:19.010394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:19.010950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:19.010950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:19.010950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:19.010950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:19.011194 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:19.011469 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:19.011581 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:19.011787 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:19.012267 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:19.012597 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:19.012843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:19.013551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:19.013551 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:19.013551 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:19.013551 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:19.013551 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:19.013551 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:19.013551 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:19.013551 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:19.014405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:19.014405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:19.014405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:19.014405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:19.014405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:19.014405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:38:19.014405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:19.014405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:19.014405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:19.014405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:19.014405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:19.014405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:19.014834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:19.014834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:19.014834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:19.014834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:19.014834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:19.014834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:19.014834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:19.014834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:19.014834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:19.014834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:19.014834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:19.014834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:19.014834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:19.015225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:19.015225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:19.015413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:19.015413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:19.015413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:19.015413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:19.015413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:19.015413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:19.015413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:19.015413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:19.015413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:19.015413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:19.015413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:19.015413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:19.015413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:19.015413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:19.015413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:19.015413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:19.015990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:38:19.015990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:19.015990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:19.015990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:19.015990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:19.015990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:19.015990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:19.015990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:19.015990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:19.015990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:19.015990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:19.015990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:19.015990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:19.015990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:19.015990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:19.015990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:19.015990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:19.015990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:19.015990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:19.016521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:19.016521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:19.016521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:23.726922 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:23.728030 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:23.728030 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:23.728424 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:23.765667 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:23.766137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:23.767049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:23.767049 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:23.767049 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:23.767049 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:23.767049 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:23.767049 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:23.767049 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:23.767049 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:23.767844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:23.767844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:23.767844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:23.767844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:23.767844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:23.767844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:38:23.767844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:23.767844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:23.767844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:23.767844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:23.767844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:23.767844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:23.768235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:23.768235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:23.768235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:23.768235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:23.768235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:23.768235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:23.768235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:23.768235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:23.768235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:23.768235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:23.768235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:23.768235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:23.768235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:23.768600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:23.768600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:23.768600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:23.768600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:23.768600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:23.768600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:23.768600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:23.768600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:23.768600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:23.768600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:23.768600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:23.768600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:23.768600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:23.768600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:23.768600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:23.769961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:23.769961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:23.769961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:23.769961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:38:23.769961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:23.769961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:23.769961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:23.769961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:23.769961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:23.769961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:23.769961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:23.769961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:23.769961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:23.769961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:23.769961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:23.769961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:23.769961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:23.769961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:23.769961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:23.771161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:23.771161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:23.771161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:23.771161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:23.771161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:23.771161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:23.956805 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:23.957444 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:23.957625 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:23.958852 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:23.959261 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:23.959564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:23.960558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:23.960558 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:23.960558 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:23.960558 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:23.960558 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:23.960558 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:23.960558 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:23.960558 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:23.962053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:23.962053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:23.962053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:23.962053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:23.962053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:23.962053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:38:23.962053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:23.962053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:23.962053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:23.962053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:23.962053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:23.962053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:23.963078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:23.963078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:23.963078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:23.963078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:23.963078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:23.963078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:23.963078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:23.963078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:23.963078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:23.963078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:23.963078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:23.963078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:23.963078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:23.963514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:23.963514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:23.963841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:23.963841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:23.963841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:23.963841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:23.963841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:23.963841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:23.963841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:23.963841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:23.963841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:23.963841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:23.963841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:23.963841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:23.963841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:23.963841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:23.963841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:23.963841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:23.964639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:38:23.964639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:23.964639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:23.964639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:23.964639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:23.964639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:23.964639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:23.964639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:23.964639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:23.964639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:23.964639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:23.964639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:23.964639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:23.964639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:23.964639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:23.964639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:23.964639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:23.964639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:23.964639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:23.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:23.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:23.965134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:23.996970 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:23.997285 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:23.997421 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:23.997580 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:38:23.998243 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:23.998579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:23.999430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:23.999430 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:23.999430 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:23.999430 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:23.999430 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:23.999430 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:23.999430 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:23.999430 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:24.000451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:24.000451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:38:24.000451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:24.000451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:24.000451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:24.000451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:24.000451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:24.000451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:24.000451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:24.000451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:24.000451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:24.000451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:38:24.000451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:24.000451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:38:24.000451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:38:24.002153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:24.002153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:38:24.002153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:24.002153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:38:24.002153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:38:24.002153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:24.002153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:38:24.002153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:24.002153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:38:24.002153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:24.002153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:24.002153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:24.002153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:24.002153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:24.002942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:38:24.002942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:38:24.002942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:38:24.002942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:24.002942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:24.002942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:24.002942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:24.002942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:24.002942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:24.002942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:24.002942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:24.002942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:24.002942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:24.003484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:24.003484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:24.003484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:24.003484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:24.003484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:24.003484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:24.003484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:24.003484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:24.003484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:24.003484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:24.003484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:24.003484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:24.004079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:24.004079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:24.004079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:24.004079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:24.004079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:24.004079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:24.004079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:24.004079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:24.004079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:24.004079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:24.004079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:24.004079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:24.004079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:24.004079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:24.004079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:24.004079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:24.004640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:38:24.004640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:38:24.004640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:38:24.004640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:24.004640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:24.004640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:24.004640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:24.004640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:24.004640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:24.004640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:24.004640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:24.004640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:24.004640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:24.004640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:24.004640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:24.004640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:24.004640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:24.004640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:24.006079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:24.006079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:24.006079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:24.006079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:24.006079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:24.006079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:24.006079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:24.006079 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:24.006079 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:24.006079 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:24.006619 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:24.006619 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:24.006619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:24.006619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:24.006619 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:24.006619 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:24.006619 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:24.006619 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:24.006619 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:24.006619 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:24.006619 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:24.007335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:24.007335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:38:24.007335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:24.007335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:24.007335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:24.007335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:24.007335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:24.007335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:24.007335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:24.007335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:24.007335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:24.007335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:38:24.007335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:24.007335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:38:24.007335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:38:24.008186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:24.008186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:38:24.008186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:24.008186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:38:24.008186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:38:24.008186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:24.008186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:38:24.008186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:24.008186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:38:24.008640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:24.008640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:24.008640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:24.008640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:24.008640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:24.008640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:38:24.008640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:38:24.008640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:24.008640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:24.008640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:24.008640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:24.009313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:24.009313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:24.009313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:24.009313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:24.009313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:24.009313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:24.009313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:24.009313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:24.009313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:24.009313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:24.009313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:24.009313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:24.009313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:24.009313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:24.010291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:24.010291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:24.010291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:24.010291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:24.010291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:24.010291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:24.010291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:24.010291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:24.010291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:24.010291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:24.010291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:24.010291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:24.010291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:24.010291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:24.011109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:24.011109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:24.011109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:24.011109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:24.011109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:24.011109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:24.011109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:38:24.011109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:38:24.011109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:24.011109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:24.011109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:24.011109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:24.011109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:24.011109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:24.011109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:24.011109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:24.011109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:24.011109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:24.011109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:24.011785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:24.011785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:24.011785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:24.011785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:24.011785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:24.011785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:24.011785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:24.011785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:24.011785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:24.011785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:24.011785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:24.011785 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:24.012220 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:24.012220 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:24.012220 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:24.012635 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:24.012943 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:24.013141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:24.013634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:24.013634 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:24.013634 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:24.013634 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:24.013634 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:24.013634 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:24.013634 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:24.013634 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:24.014466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:24.014466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:24.014466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:24.014466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:24.014466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:24.014466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:38:24.014466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:24.014466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:24.014466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:24.014466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:24.014466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:24.014466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:24.014817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:24.014817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:24.014817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:24.014817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:24.014817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:24.014817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:24.014817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:24.014817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:24.014817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:24.014817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:24.014817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:24.014817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:24.014817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:24.015065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:24.015065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:24.015477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:24.015477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:24.015477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:24.015477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:24.015477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:24.015477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:24.015477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:24.015477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:24.015477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:24.015477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:24.015477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:24.015477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:24.015477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:24.015477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:24.015477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:24.015477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:24.015854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:38:24.015854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:24.015854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:24.015854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:24.015854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:24.015854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:24.015854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:24.015854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:24.015854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:24.015854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:24.015854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:24.015854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:24.015854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:24.015854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:24.015854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:24.015854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:24.015854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:24.015854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:24.015854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:24.016221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:24.016221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:24.016221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:28.727535 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:28.728309 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:28.728309 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:28.729168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:28.767244 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:28.767518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:28.769379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:28.769379 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:28.769379 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:28.769379 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:28.769379 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:28.769379 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:28.769379 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:28.769379 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:28.771425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:28.771425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:28.771425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:28.771425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:28.771425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:28.771425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:38:28.771425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:28.771425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:28.771425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:28.771425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:28.771425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:28.771425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:28.772071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:28.772071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:28.772071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:28.772071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:28.772071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:28.772071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:28.772071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:28.772071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:28.772071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:28.772071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:28.772071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:28.772071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:28.772071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:28.772667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:28.772667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:28.773709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:28.773709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:28.773709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:28.773709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:28.773709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:28.773709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:28.773709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:28.773709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:28.773709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:28.773709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:28.773709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:28.773709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:28.773709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:28.773709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:28.773709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:28.773709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:28.776442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:38:28.776442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:28.776442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:28.776442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:28.776442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:28.776442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:28.776442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:28.776442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:28.776442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:28.776442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:28.776442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:28.776442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:28.776442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:28.776442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:28.776442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:28.776442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:28.776442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:28.776442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:28.776442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:28.776971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:28.776971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:28.776971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:28.957543 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:28.958419 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:28.958610 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:28.959923 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:28.960205 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:28.960432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:28.961472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:28.961472 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:28.961472 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:28.961472 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:28.961472 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:28.961472 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:28.961472 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:28.961472 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:28.963030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:28.963030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:28.963030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:28.963030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:28.963030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:28.963030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:38:28.963030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:28.963030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:28.963030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:28.963030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:28.963030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:28.963030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:28.963653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:28.963653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:28.963653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:28.963653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:28.963653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:28.963653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:28.963653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:28.963653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:28.963653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:28.963653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:28.963653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:28.963653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:28.963653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:28.964311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:28.964311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:28.964311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:28.964311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:28.964311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:28.964311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:28.964311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:28.964311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:28.964311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:28.964311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:28.964311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:28.964311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:28.964311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:28.964311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:28.964311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:28.965010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:28.965010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:28.965010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:28.965010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:38:28.965010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:28.965010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:28.965010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:28.965010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:28.965010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:28.965010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:28.965010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:28.965010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:28.965010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:28.965010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:28.965010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:28.965010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:28.965010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:28.965010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:28.965010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:28.965858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:28.965858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:28.965858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:28.965858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:28.965858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:28.965858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:28.997524 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:28.997985 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:28.998168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:28.998364 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:38:28.999485 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:28.999978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:29.001132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:29.001132 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:29.001132 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:29.001132 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:29.001132 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:29.001132 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:29.001132 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:29.001132 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:29.002348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:29.002348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:38:29.002348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:29.002348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:29.002348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:29.002348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:29.002348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:29.002348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:29.002348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:29.002348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:29.002348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:29.002348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:38:29.002348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:29.002348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:38:29.002348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:38:29.003072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:29.003072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:38:29.003072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:29.003072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:38:29.003072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:38:29.003072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:29.003072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:38:29.003072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:29.003072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:38:29.003896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:29.003896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:29.003896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:29.003896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:29.003896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:29.003896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:38:29.003896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:38:29.003896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:38:29.003896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:29.003896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:29.003896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:29.003896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:29.004528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:29.004528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:29.004528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:29.004528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:29.004528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:29.004528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:29.004528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:29.004528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:29.004528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:29.004528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:29.004528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:29.004528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:29.004528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:29.004528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:29.005199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:29.005199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:29.005199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:29.005199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:29.005570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:29.005570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:29.005570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:29.005570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:29.005570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:29.005570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:29.005570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:29.005570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:29.005570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:29.005570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:29.005570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:29.005570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:29.005570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:29.005570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:29.005570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:29.005570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:29.006344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:38:29.006344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:38:29.006344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:38:29.006344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:29.006344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:29.006344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:29.006344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:29.006344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:29.006344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:29.006344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:29.006344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:29.006344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:29.006344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:29.006344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:29.006344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:29.006344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:29.006344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:29.006344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:29.007318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:29.007318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:29.007318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:29.007318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:29.007318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:29.007318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:29.007318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:29.007318 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:29.007318 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:29.007318 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:29.007839 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:29.007839 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:29.007839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:29.008541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:29.008541 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:29.008541 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:29.008541 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:29.008541 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:29.008541 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:29.008541 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:29.008541 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:29.009297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:29.009297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:38:29.009297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:29.009297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:29.009297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:29.009297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:29.009297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:29.009297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:29.009297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:29.009297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:29.009297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:29.009297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:38:29.009297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:29.009297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:38:29.009297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:38:29.009705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:29.009705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:38:29.009705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:29.009705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:38:29.009705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:38:29.009705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:29.009705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:38:29.009705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:29.009705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:38:29.010187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:29.010187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:29.010187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:29.010187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:29.010187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:29.010187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:38:29.010187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:38:29.010187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:29.010187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:29.010187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:29.010187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:29.010538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:29.010538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:29.010538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:29.010538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:29.010538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:29.010538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:29.010538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:29.010538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:29.010538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:29.010538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:29.010538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:29.010538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:29.010538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:29.010538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:29.010927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:29.010927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:29.010927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:29.010927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:29.011361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:29.011361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:29.011361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:29.011361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:29.011361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:29.011361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:29.011361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:29.011361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:29.011361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:29.011361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:29.011361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:29.011361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:29.011361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:29.011361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:29.011361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:29.011361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:29.011361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:38:29.011842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:38:29.011842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:29.011842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:29.011842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:29.011842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:29.011842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:29.011842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:29.011842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:29.011842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:29.011842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:29.011842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:29.011842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:29.011842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:29.011842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:29.011842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:29.011842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:29.011842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:29.011842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:29.011842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:29.012364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:29.012364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:29.012364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:29.012364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:29.012364 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:29.012513 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:29.012642 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:29.012823 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:29.013333 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:29.013717 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:29.013998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:29.014832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:29.014832 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:29.014832 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:29.014832 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:29.014832 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:29.014832 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:29.014832 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:29.014832 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:29.015684 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:29.015684 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:29.015684 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:29.015684 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:29.015684 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:29.015684 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:38:29.015684 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:29.015684 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:29.015684 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:29.015684 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:29.015684 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:29.015684 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:29.016023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:29.016023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:29.016023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:29.016023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:29.016023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:29.016023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:29.016023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:29.016023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:29.016023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:29.016023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:29.016023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:29.016023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:29.016023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:29.016404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:29.016404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:29.016640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:29.016640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:29.016640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:29.016640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:29.016640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:29.016640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:29.016640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:29.016640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:29.016640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:29.016640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:29.016640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:29.016640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:29.016640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:29.016640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:29.016640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:29.016640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:29.017084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:38:29.017084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:29.017084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:29.017084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:29.017084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:29.017084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:29.017084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:29.017084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:29.017084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:29.017084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:29.017084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:29.017084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:29.017084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:29.017084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:29.017084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:29.017084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:29.017084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:29.017084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:29.017084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:29.017593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:29.017593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:29.017593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:33.728283 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:33.729435 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:33.729435 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:33.729938 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:33.764110 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:33.764276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:33.764732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:33.764732 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:33.764732 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:33.764732 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:33.764732 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:33.764732 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:33.764732 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:33.764732 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:33.765569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:33.765569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:33.765569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:33.765569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:33.765569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:33.765569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:38:33.765569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:33.765569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:33.765569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:33.765569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:33.765569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:33.765569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:33.765938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:33.765938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:33.765938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:33.765938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:33.765938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:33.765938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:33.765938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:33.765938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:33.765938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:33.765938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:33.765938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:33.765938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:33.765938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:33.766294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:33.766294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:33.766611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:33.766611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:33.766611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:33.766611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:33.766611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:33.766611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:33.766611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:33.766611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:33.766611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:33.766611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:33.766611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:33.766611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:33.766611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:33.766611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:33.766611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:33.766611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:33.767031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:38:33.767031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:33.767031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:33.767031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:33.767031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:33.767031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:33.767031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:33.767031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:33.767031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:33.767031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:33.767031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:33.767031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:33.767031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:33.767031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:33.767031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:33.767031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:33.767031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:33.767031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:33.767031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:33.767503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:33.767503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:33.767503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:33.958439 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:33.959129 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:33.959315 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:33.960168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:33.960427 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:33.960641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:33.961623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:33.961623 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:33.961623 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:33.961623 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:33.961623 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:33.961623 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:33.961623 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:33.961623 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:33.963029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:33.963029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:33.963029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:33.963029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:33.963029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:33.963029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:38:33.963029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:33.963029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:33.963029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:33.963029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:33.963029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:33.963029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:33.963585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:33.963585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:33.963585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:33.963585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:33.963585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:33.963585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:33.963585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:33.963585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:33.963585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:33.963585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:33.963585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:33.963585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:33.963585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:33.964340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:33.964340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:33.965199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:33.965199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:33.965199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:33.965199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:33.965199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:33.965199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:33.965199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:33.965199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:33.965199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:33.965199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:33.965199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:33.965199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:33.965199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:33.965199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:33.965199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:33.965199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:33.965972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:38:33.965972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:33.965972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:33.965972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:33.965972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:33.965972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:33.965972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:33.965972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:33.965972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:33.965972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:33.965972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:33.965972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:33.965972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:33.965972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:33.965972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:33.965972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:33.965972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:33.965972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:33.965972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:33.966811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:33.966811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:33.966811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:33.998188 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:33.998636 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:33.998827 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:33.999015 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:38:33.999389 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:33.999654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:34.000344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:34.000344 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:34.000344 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:34.000344 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:34.000344 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:34.000344 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:34.000344 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:34.000344 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:34.001555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:34.001555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:38:34.001555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:34.001555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:34.001555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:34.001555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:34.001555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:34.001555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:34.001555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:34.001555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:34.001555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:34.001555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:38:34.001555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:34.001555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:38:34.001555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:38:34.002357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:34.002357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:38:34.002357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:34.002357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:38:34.002357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:38:34.002357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:34.002357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:38:34.002357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:34.002357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:38:34.002823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:34.002823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:34.002823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:34.002823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:34.002823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:34.002823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:38:34.002823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:38:34.002823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:38:34.002823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:34.002823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:34.002823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:34.002823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:34.003338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:34.003338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:34.003338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:34.003338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:34.003338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:34.003338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:34.003338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:34.003338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:34.003338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:34.003338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:34.003338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:34.003338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:34.003338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:34.003338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:34.003902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:34.003902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:34.003902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:34.003902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:34.003902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:34.003902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:34.003902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:34.003902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:34.003902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:34.003902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:34.003902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:34.003902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:34.003902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:34.003902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:34.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:34.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:34.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:34.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:34.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:34.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:34.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:38:34.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:38:34.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:38:34.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:34.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:34.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:34.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:34.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:34.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:34.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:34.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:34.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:34.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:34.005633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:34.005633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:34.005633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:34.005633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:34.005633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:34.005633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:34.005633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:34.005633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:34.005633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:34.005633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:34.005633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:34.005633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:34.006353 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:34.006761 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:34.006880 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:34.007011 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:34.007264 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:34.007493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:34.007975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:34.007975 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:34.007975 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:34.007975 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:34.007975 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:34.007975 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:34.007975 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:34.007975 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:34.008735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:34.008735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:38:34.008735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:34.008735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:34.008735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:34.008735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:34.008735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:34.008735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:34.008735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:34.008735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:34.008735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:34.008735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:38:34.008735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:34.008735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:38:34.008735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:38:34.009442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:34.009442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:38:34.009442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:34.009442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:38:34.009442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:38:34.009442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:34.009442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:38:34.009442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:34.009442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:38:34.009991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:34.009991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:34.009991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:34.009991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:34.009991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:34.009991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:38:34.009991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:38:34.009991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:34.009991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:34.009991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:34.009991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:34.010388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:34.010388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:34.010388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:34.010388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:34.010388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:34.010388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:34.010388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:34.010388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:34.010388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:34.010388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:34.010388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:34.010388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:34.010388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:34.010388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:34.010784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:34.010784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:34.010784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:34.010784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:34.010918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:34.010918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:34.010918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:34.010918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:34.010918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:34.010918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:34.010918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:34.010918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:34.010918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:34.010918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:34.010918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:34.010918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:34.010918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:34.010918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:34.010918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:34.010918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:34.010918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:38:34.011388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:38:34.011388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:34.011388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:34.011388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:34.011388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:34.011388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:34.011388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:34.011388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:34.011388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:34.011388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:34.011388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:34.011388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:34.011388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:34.011388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:34.011388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:34.011388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:34.011388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:34.011388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:34.011388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:34.012193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:34.012193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:34.012193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:34.012193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:34.012556 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:34.012842 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:34.012963 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:34.013156 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:34.013656 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:34.013913 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:34.014078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:34.015810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:34.015810 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:34.015810 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:34.015810 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:34.015810 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:34.015810 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:34.015810 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:34.015810 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:34.015810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:34.015810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:34.015810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:34.015810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:34.015810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:34.015810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:38:34.015810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:34.016799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:34.016799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:34.016799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:34.016799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:34.016799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:34.016799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:34.016799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:34.016799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:34.016799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:34.016799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:34.016799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:34.016799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:34.016799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:34.016799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:34.017542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:34.017542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:34.017542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:34.017542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:34.017542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:34.017542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:34.017542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:34.017542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:34.017542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:34.017542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:34.017542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:34.017542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:34.017542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:34.017542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:34.018243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:34.018243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:34.018243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:34.018243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:34.018243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:34.018243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:34.018243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:34.018243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:34.018243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:38:34.018243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:34.018243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:34.018243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:34.018243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:34.018243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:34.018243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:34.018243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:34.018243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:34.018243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:34.018243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:34.018243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:34.019603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:34.019603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:34.019603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:34.019603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:34.019603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:34.019603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:34.019603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:34.019603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:34.019603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:34.019603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:38.728569 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:38.729237 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:38.729441 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:38.730590 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:38.765532 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:38.766008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:38.766920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:38.766920 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:38.766920 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:38.766920 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:38.766920 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:38.766920 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:38.766920 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:38.766920 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:38.767974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:38.767974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:38.767974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:38.767974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:38.767974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:38.767974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:38:38.767974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:38.767974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:38.767974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:38.767974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:38.767974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:38.767974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:38.769082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:38.769082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:38.769082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:38.769082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:38.769082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:38.769082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:38.769082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:38.769082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:38.769082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:38.769082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:38.769082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:38.769082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:38.769082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:38.769763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:38.769763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:38.769763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:38.769763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:38.769763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:38.769763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:38.769763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:38.769763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:38.769763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:38.769763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:38.769763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:38.769763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:38.769763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:38.769763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:38.769763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:38.770528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:38.770528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:38.770528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:38.770528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:38:38.770528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:38.770528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:38.770528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:38.770528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:38.770528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:38.770528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:38.770528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:38.770528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:38.770528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:38.770528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:38.770528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:38.770528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:38.770528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:38.770528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:38.770528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:38.771483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:38.771483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:38.771483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:38.771483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:38.771483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:38.771483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:38.959336 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:38.959932 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:38.960131 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:38.961205 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:38.961523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:38.961523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:38.962250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:38.962250 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:38.962250 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:38.962250 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:38.962250 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:38.962250 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:38.962250 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:38.962250 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:38.964217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:38.964217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:38.964217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:38.964217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:38.964217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:38.964217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:38:38.964217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:38.964217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:38.964217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:38.964217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:38.964217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:38.964217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:38.965327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:38.965327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:38.965327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:38.965327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:38.965327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:38.965327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:38.965327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:38.965327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:38.965327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:38.965327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:38.965327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:38.965327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:38.965327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:38.966144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:38.966144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:38.966144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:38.966144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:38.966144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:38.966144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:38.966144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:38.966144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:38.966144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:38.966144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:38.966144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:38.966144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:38.966144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:38.966144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:38.966144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:38.967493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:38.967493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:38.967493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:38.967493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:38:38.967493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:38.967493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:38.967493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:38.967493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:38.967493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:38.967493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:38.967493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:38.967493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:38.967493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:38.967493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:38.967493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:38.967493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:38.967493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:38.967493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:38.967493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:38.970227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:38.970227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:38.970227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:38.970227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:38.970227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:38.970227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:38.998827 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:38.999145 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:38.999292 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:38.999496 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:38:39.000100 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:39.000439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:39.001347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:39.001347 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:39.001347 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:39.001347 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:39.001347 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:39.001347 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:39.001347 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:39.001347 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:39.002554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:39.002554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:38:39.002554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:39.002554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:39.002554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:39.002554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:39.002554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:39.002554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:39.002554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:39.002554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:39.002554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:39.002554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:38:39.002554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:39.002554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:38:39.002554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:38:39.003740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:39.003740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:38:39.003740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:39.003740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:38:39.003740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:38:39.003740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:39.003740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:38:39.003740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:39.003740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:38:39.004260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:39.004260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:39.004260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:39.004260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:39.004260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:39.004260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:38:39.004260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:38:39.004260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:38:39.004260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:39.004260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:39.004260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:39.004260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:39.004907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:39.004907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:39.004907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:39.004907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:39.004907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:39.004907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:39.004907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:39.004907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:39.004907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:39.004907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:39.004907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:39.004907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:39.004907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:39.004907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:39.005580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:39.005580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:39.005580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:39.005580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:39.005580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:39.005580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:39.005580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:39.005580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:39.005580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:39.005580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:39.005580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:39.005580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:39.005580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:39.005580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:39.007624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:39.007624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:39.007624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:39.007624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:39.007624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:39.007624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:39.007624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:38:39.007624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:38:39.007624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:38:39.007624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:39.007624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:39.007624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:39.007624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:39.007624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:39.007624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:39.007624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:39.007624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:39.007624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:39.007624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:39.009608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:39.009608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:39.009608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:39.009608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:39.009608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:39.009608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:39.009608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:39.009608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:39.009608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:39.009608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:39.009608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:39.009608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:39.009608 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:39.009608 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:39.010553 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:39.010553 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:39.010553 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:39.010553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:39.010553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:39.010553 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:39.010553 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:39.010553 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:39.010553 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:39.010553 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:39.010553 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:39.010553 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:39.011416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:39.011416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:38:39.011416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:39.011416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:39.011416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:39.011416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:39.011416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:39.011416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:39.011416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:39.011416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:39.011416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:39.011416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:38:39.011416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:39.011416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:38:39.011416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:38:39.012232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:39.012232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:38:39.012232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:39.012232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:38:39.012232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:38:39.012232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:39.012232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:38:39.012232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:39.012232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:38:39.012232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:39.012232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:39.012232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:39.012232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:39.012232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:39.012785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:38:39.012785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:38:39.012785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:39.012785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:39.012785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:39.012785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:39.012785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:39.012785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:39.012785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:39.012785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:39.012785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:39.012785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:39.012785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:39.014554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:39.014554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:39.014554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:39.014554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:39.014554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:39.014554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:39.014554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:39.014554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:39.014554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:39.014554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:39.014554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:39.014554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:39.015538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:39.015538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:39.015538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:39.015538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:39.015538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:39.015538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:39.015538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:39.015538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:39.015538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:39.015538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:39.015538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:39.015538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:39.015538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:39.015538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:39.015538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:39.015538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:38:39.015538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:38:39.015538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:39.015538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:39.017356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:39.017356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:39.017356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:39.017356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:39.017356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:39.017356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:39.017356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:39.017356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:39.017356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:39.017356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:39.017356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:39.017356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:39.017356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:39.017356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:39.017356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:39.017356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:39.017356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:39.017356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:39.017356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:39.017356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:39.018112 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:39.018112 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:39.018112 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:39.018112 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:39.018112 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:39.018112 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:39.018719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:39.018719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:39.018719 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:39.018719 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:39.018719 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:39.018719 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:39.018719 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:39.018719 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:39.018719 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:39.018719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:39.018719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:39.018719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:39.018719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:39.018719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:39.018719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:38:39.019174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:39.019174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:39.019174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:39.019174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:39.019174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:39.019174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:39.019174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:39.019174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:39.019174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:39.019174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:39.019174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:39.019174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:39.019174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:39.019174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:39.019564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:39.019564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:39.019564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:39.019564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:39.019564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:39.019564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:39.019564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:39.019564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:39.019564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:39.019564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:39.019564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:39.019564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:39.019564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:39.019955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:39.019955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:39.019955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:39.019955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:39.019955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:39.019955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:39.019955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:39.019955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:39.019955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:39.019955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:39.019955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:38:39.019955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:39.019955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:39.019955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:39.019955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:39.019955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:39.019955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:39.019955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:39.019955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:39.019955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:39.020474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:39.020474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:39.020474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:39.020474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:39.020474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:39.020474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:39.020474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:39.020474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:39.020474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:39.020474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:39.020474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:39.020474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:43.729271 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:43.730306 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:43.730306 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:43.731302 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:43.765471 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:43.765916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:43.766876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:43.766876 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:43.766876 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:43.766876 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:43.766876 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:43.766876 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:43.766876 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:43.766876 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:43.767874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:43.767874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:43.767874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:43.767874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:43.767874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:43.767874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:38:43.767874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:43.767874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:43.767874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:43.767874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:43.767874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:43.767874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:43.768418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:43.768418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:43.768418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:43.768418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:43.768418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:43.768418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:43.768418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:43.768418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:43.768418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:43.768418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:43.768418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:43.768418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:43.768418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:43.769329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:43.769329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:43.769329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:43.769329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:43.769329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:43.769329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:43.769329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:43.769329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:43.769329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:43.769329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:43.769329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:43.769329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:43.769329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:43.769329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:43.769329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:43.770912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:43.770912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:43.770912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:43.770912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:38:43.770912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:43.770912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:43.770912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:43.770912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:43.770912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:43.770912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:43.770912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:43.770912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:43.770912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:43.770912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:43.770912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:43.770912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:43.770912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:43.770912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:43.770912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:43.772091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:43.772091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:43.772091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:43.772091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:43.772091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:43.772091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:43.960004 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:43.960647 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:43.960876 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:43.962065 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:43.962406 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:43.962640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:43.963571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:43.963571 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:43.963571 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:43.963571 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:43.963571 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:43.963571 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:43.963571 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:43.963571 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:43.965154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:43.965154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:43.965154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:43.965154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:43.965154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:43.965154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:38:43.965154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:43.965154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:43.965154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:43.965154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:43.965154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:43.965154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:43.965754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:43.965754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:43.965754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:43.965754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:43.965754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:43.965754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:43.965754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:43.965754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:43.965754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:43.965754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:43.965754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:43.965754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:43.965754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:43.966526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:43.966526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:43.967348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:43.967348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:43.967348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:43.967348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:43.967348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:43.967348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:43.967348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:43.967348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:43.967348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:43.967348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:43.967348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:43.967348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:43.967348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:43.967348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:43.967348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:43.967348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:43.968508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:38:43.968508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:43.968508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:43.968508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:43.968508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:43.968508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:43.968508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:43.968508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:43.968508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:43.968508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:43.968508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:43.968508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:43.968508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:43.968508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:43.968508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:43.968508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:43.968508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:43.968508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:43.968508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:43.969557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:43.969557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:43.969557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:43.999465 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:43.999878 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:44.000086 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:44.000424 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:38:44.001273 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:44.001937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:44.002838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:44.002838 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:44.002838 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:44.002838 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:44.002838 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:44.002838 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:44.002838 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:44.002838 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:44.004505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:44.004505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:38:44.004505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:44.004505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:44.004505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:44.004505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:44.004505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:44.004505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:44.004505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:44.004505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:44.004505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:44.004505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:38:44.004505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:44.004505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:38:44.004505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:38:44.005344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:44.005344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:38:44.005344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:44.005344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:38:44.005344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:38:44.005344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:44.005344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:38:44.005344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:44.005344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:38:44.005824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:44.005824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:44.005824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:44.005824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:44.005824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:44.005824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:38:44.005824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:38:44.005824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:38:44.005824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:44.005824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:44.005824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:44.005824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:44.006385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:44.006385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:44.006385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:44.006385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:44.006385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:44.006385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:44.006385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:44.006385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:44.006385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:44.006385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:44.006385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:44.006385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:44.006385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:44.006385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:44.007081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:44.007081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:44.007081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:44.007081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:44.007081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:44.007081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:44.007081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:44.007081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:44.007081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:44.007081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:44.007081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:44.007081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:44.007081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:44.007081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:44.007826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:44.007826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:44.007826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:44.007826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:44.007826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:44.007826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:44.007826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:38:44.007826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:38:44.007826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:38:44.007826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:44.007826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:44.007826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:44.007826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:44.007826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:44.007826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:44.007826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:44.007826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:44.007826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:44.007826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:44.008758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:44.008758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:44.008758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:44.008758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:44.008758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:44.008758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:44.008758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:44.008758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:44.008758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:44.008758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:44.008758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:44.008758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:44.008758 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:44.008758 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:44.009327 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:44.009327 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:44.009327 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:44.009327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:44.009878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:44.009878 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:44.009878 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:44.009878 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:44.009878 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:44.009878 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:44.009878 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:44.009878 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:44.010650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:44.010650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:38:44.010650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:44.010650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:44.010650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:44.010650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:44.010650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:44.010650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:44.010650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:44.010650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:44.010650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:44.010650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:38:44.010650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:44.010650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:38:44.010650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:38:44.011061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:44.011061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:38:44.011061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:44.011061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:38:44.011061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:38:44.011061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:44.011061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:38:44.011061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:44.011061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:38:44.011619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:44.011619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:44.011619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:44.011619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:44.011619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:44.011619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:38:44.011619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:38:44.011619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:44.011619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:44.011619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:44.011619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:44.011977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:44.011977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:44.011977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:44.011977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:44.011977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:44.011977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:44.011977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:44.011977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:44.011977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:44.011977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:44.011977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:44.011977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:44.011977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:44.011977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:44.012400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:44.012400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:44.012400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:44.012400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:44.013061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:44.013061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:44.013061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:44.013061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:44.013061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:44.013061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:44.013061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:44.013061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:44.013061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:44.013061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:44.013061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:44.013061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:44.013061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:44.013061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:44.013061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:44.013061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:44.013061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:38:44.013589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:38:44.013589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:44.013589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:44.013589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:44.013589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:44.013589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:44.013589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:44.013589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:44.013589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:44.013589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:44.013589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:44.013589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:44.013589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:44.013589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:44.013589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:44.013589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:44.013589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:44.013589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:44.013589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:44.014130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:44.014130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:44.014130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:44.014130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:44.014130 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:44.014130 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:44.014130 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:44.014368 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:44.014555 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:44.014821 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:44.014973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:44.015434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:44.015434 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:44.015434 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:44.015434 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:44.015434 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:44.015434 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:44.015434 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:44.015434 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:44.016385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:44.016385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:44.016385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:44.016385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:44.016385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:44.016385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:38:44.016385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:44.016385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:44.016385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:44.016385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:44.016385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:44.016385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:44.016782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:44.016782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:44.016782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:44.016782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:44.016782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:44.016782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:44.016782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:44.016782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:44.016782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:44.016782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:44.016782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:44.016782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:44.016782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:44.017138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:44.017138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:44.017445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:44.017445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:44.017445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:44.017445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:44.017445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:44.017445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:44.017445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:44.017445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:44.017445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:44.017445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:44.017445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:44.017445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:44.017445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:44.017445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:44.017445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:44.017445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:44.017924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:38:44.017924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:44.017924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:44.017924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:44.017924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:44.017924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:44.017924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:44.017924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:44.017924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:44.017924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:44.017924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:44.017924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:44.017924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:44.017924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:44.017924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:44.017924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:44.017924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:44.017924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:44.017924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:44.018472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:44.018472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:44.018472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:48.730029 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:48.730948 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:48.730948 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:48.732376 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:48.768135 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:48.768402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:48.769182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:48.769182 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:48.769182 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:48.769182 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:48.769182 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:48.769182 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:48.769182 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:48.769182 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:48.769784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:48.769784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:48.769784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:48.769784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:48.769784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:48.769784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:38:48.769784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:48.769784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:48.769784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:48.769784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:48.769784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:48.769784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:48.770215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:48.770215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:48.770215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:48.770215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:48.770215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:48.770215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:48.770215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:48.770215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:48.770215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:48.770215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:48.770215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:48.770215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:48.770215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:48.770592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:48.770592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:48.770592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:48.770592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:48.770592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:48.770592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:48.770592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:48.770592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:48.770592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:48.770592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:48.770592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:48.770592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:48.770592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:48.770592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:48.770592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:48.771077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:48.771077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:48.771077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:48.771077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:38:48.771077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:48.771077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:48.771077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:48.771077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:48.771077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:48.771077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:48.771077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:48.771077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:48.771077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:48.771077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:48.771077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:48.771077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:48.771077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:48.771077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:48.771077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:48.772237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:48.772237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:48.772237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:48.772237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:48.772237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:48.772237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:48.960796 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:48.961176 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:48.961352 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:48.962277 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:48.962544 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:48.962756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:48.963605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:48.963605 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:48.963605 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:48.963605 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:48.963605 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:48.963605 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:48.963605 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:48.963605 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:48.964582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:48.964582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:48.964582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:48.964582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:48.964582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:48.964582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:38:48.964582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:48.964582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:48.964582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:48.964582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:48.964582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:48.964582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:48.965365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:48.965365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:48.965365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:48.965365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:48.965365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:48.965365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:48.965365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:48.965365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:48.965365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:48.965365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:48.965365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:48.965365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:48.965365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:48.966135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:48.966135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:48.966135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:48.966135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:48.966135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:48.966135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:48.966135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:48.966135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:48.966135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:48.966135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:48.966135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:48.966135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:48.966135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:48.966135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:48.966135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:48.966887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:48.966887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:48.966887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:48.966887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:38:48.966887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:48.966887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:48.966887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:48.966887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:48.966887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:48.966887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:48.966887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:48.966887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:48.966887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:48.966887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:48.966887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:48.966887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:48.966887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:48.966887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:48.966887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:48.967607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:48.967607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:48.967607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:48.967607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:48.967607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:48.967607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:48.999822 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:49.000123 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:49.000321 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:49.000516 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:38:49.001026 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:49.001236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:49.001875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:49.001875 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:49.001875 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:49.001875 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:49.001875 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:49.001875 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:49.001875 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:49.001875 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:49.003204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:49.003204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:38:49.003204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:49.003204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:49.003204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:49.003204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:49.003204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:49.003204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:49.003204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:49.003204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:49.003204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:49.003204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:38:49.003204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:49.003204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:38:49.003204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:38:49.003646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:49.003646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:38:49.003646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:49.003646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:38:49.003646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:38:49.003646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:49.003646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:38:49.003646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:49.003646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:38:49.004598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:49.004598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:49.004598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:49.004598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:49.004598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:49.004598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:38:49.004598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:38:49.004598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:38:49.004598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:49.004598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:49.004598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:49.004598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:49.004996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:49.004996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:49.004996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:49.004996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:49.004996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:49.004996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:49.004996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:49.004996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:49.004996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:49.004996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:49.004996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:49.004996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:49.004996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:49.004996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:49.005353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:49.005353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:49.005353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:49.005353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:49.006337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:49.006337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:49.006337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:49.006337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:49.006337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:49.006337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:49.006337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:49.006337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:49.006337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:49.006337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:49.006337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:49.006337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:49.006337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:49.006337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:49.006337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:49.006337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:49.006926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:38:49.006926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:38:49.006926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:38:49.006926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:49.006926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:49.006926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:49.006926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:49.006926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:49.006926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:49.006926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:49.006926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:49.006926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:49.006926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:49.006926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:49.006926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:49.006926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:49.006926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:49.006926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:49.007444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:49.007444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:49.007444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:49.007444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:49.007444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:49.007444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:49.007444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:49.008520 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:49.008878 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:49.009055 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:49.009250 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:49.009767 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:49.010161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:49.011386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:49.011386 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:49.011386 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:49.011386 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:49.011386 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:49.011386 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:49.011386 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:49.011386 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:49.012115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:49.012115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:38:49.012115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:49.012115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:49.012115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:49.012115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:49.012115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:49.012115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:49.012115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:49.012115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:49.012115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:49.012115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:38:49.012115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:49.012115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:38:49.012115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:38:49.012535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:49.012535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:38:49.012535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:49.012535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:38:49.012535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:38:49.012535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:49.012535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:38:49.012535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:49.012535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:38:49.012954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:49.012954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:49.012954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:49.012954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:49.012954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:49.012954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:38:49.012954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:38:49.012954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:49.012954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:49.012954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:49.012954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:49.013266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:49.013266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:49.013266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:49.013266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:49.013266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:49.013266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:49.013266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:49.013266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:49.013266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:49.013266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:49.013266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:49.013266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:49.013266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:49.013266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:49.013654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:49.013654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:49.013654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:49.013654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:49.013978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:49.013978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:49.013978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:49.013978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:49.013978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:49.013978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:49.013978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:49.013978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:49.013978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:49.013978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:49.013978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:49.013978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:49.013978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:49.013978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:49.013978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:49.013978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:49.013978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:38:49.014431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:38:49.014431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:49.014431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:49.014431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:49.014431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:49.014431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:49.014431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:49.014431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:49.014431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:49.014431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:49.014431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:49.014431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:49.014431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:49.014431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:49.014431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:49.014431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:49.014431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:49.014431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:49.014431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:49.014909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:49.014909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:49.014909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:49.014909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:49.014909 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:49.015088 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:49.015125 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:49.015269 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:49.015767 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:49.016176 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:49.016462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:49.017183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:49.017183 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:49.017183 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:49.017183 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:49.017183 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:49.017183 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:49.017183 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:49.017183 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:49.017976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:49.017976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:49.017976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:49.017976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:49.017976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:49.017976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:38:49.017976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:49.017976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:49.017976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:49.017976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:49.017976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:49.017976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:49.018329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:49.018329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:49.018329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:49.018329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:49.018329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:49.018329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:49.018329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:49.018329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:49.018329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:49.018329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:49.018329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:49.018329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:49.018329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:49.018713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:49.018713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:49.018897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:49.018897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:49.018897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:49.018897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:49.018897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:49.018897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:49.018897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:49.018897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:49.018897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:49.018897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:49.018897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:49.018897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:49.018897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:49.018897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:49.018897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:49.018897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:49.019366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:38:49.019366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:49.019366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:49.019366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:49.019366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:49.019366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:49.019366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:49.019366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:49.019366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:49.019366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:49.019366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:49.019366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:49.019366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:49.019366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:49.019366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:49.019366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:49.019366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:49.019366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:49.019366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:49.019920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:49.019920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:49.019920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:53.730665 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:53.731577 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:53.731577 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:53.732858 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:53.764889 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:53.765666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:53.766986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:53.766986 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:53.766986 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:53.766986 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:53.766986 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:53.766986 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:53.766986 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:53.766986 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:53.768059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:53.768059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:53.768059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:53.768059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:53.768059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:53.768059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:38:53.768059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:53.768059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:53.768059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:53.768059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:53.768059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:53.768059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:53.768630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:53.768630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:53.768630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:53.768630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:53.768630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:53.768630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:53.768630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:53.768630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:53.768630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:53.768630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:53.768630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:53.768630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:53.768630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:53.769057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:53.769057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:53.769149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:53.769149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:53.769149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:53.769149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:53.769149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:53.769149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:53.769149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:53.769149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:53.769149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:53.769149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:53.769149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:53.769149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:53.769149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:53.769149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:53.769149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:53.769149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:53.769796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:38:53.769796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:53.769796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:53.769796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:53.769796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:53.769796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:53.769796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:53.769796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:53.769796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:53.769796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:53.769796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:53.769796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:53.769796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:53.769796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:53.769796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:53.769796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:53.769796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:53.769796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:53.769796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:53.770532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:53.770532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:53.770532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:53.961500 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:53.962151 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:53.962314 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:53.963562 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:53.963967 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:53.964191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:53.965150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:53.965150 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:53.965150 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:53.965150 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:53.965150 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:53.965150 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:53.965150 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:53.965150 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:53.966652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:53.966652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:53.966652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:53.966652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:53.966652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:53.966652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:38:53.966652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:53.966652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:53.966652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:53.966652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:53.966652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:53.966652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:53.967044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:53.967044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:53.967044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:53.967044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:53.967044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:53.967044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:53.967044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:53.967044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:53.967044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:53.967044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:53.967044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:53.967044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:53.967044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:53.967434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:53.967434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:53.968362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:53.968362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:53.968362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:53.968362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:53.968362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:53.968362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:53.968362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:53.968362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:53.968362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:53.968362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:53.968362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:53.968362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:53.968362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:53.968362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:53.968362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:53.968362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:53.968877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:38:53.968877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:53.968877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:53.968877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:53.968877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:53.968877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:53.968877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:53.968877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:53.968877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:53.968877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:53.968877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:53.968877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:53.968877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:53.968877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:53.968877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:53.968877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:53.968877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:53.968877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:53.968877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:53.969547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:53.969547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:53.969547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:54.000247 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:54.000777 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:54.000967 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:54.001159 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:38:54.001893 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:54.002212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:54.003163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:54.003163 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:54.003163 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:54.003163 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:54.003163 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:54.003163 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:54.003163 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:54.003163 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:54.004608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:54.004608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:38:54.004608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:54.004608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:54.004608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:54.004608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:54.004608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:54.004608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:54.004608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:54.004608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:54.004608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:54.004608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:38:54.004608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:54.004608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:38:54.004608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:38:54.005538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:54.005538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:38:54.005538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:54.005538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:38:54.005538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:38:54.005538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:54.005538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:38:54.005538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:54.005538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:38:54.006080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:54.006080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:54.006080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:54.006080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:54.006080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:54.006080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:38:54.006080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:38:54.006080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:38:54.006080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:54.006080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:54.006080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:54.006080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:54.006771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:54.006771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:54.006771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:54.006771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:54.006771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:54.006771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:54.006771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:54.006771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:54.006771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:54.006771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:54.006771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:54.006771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:54.006771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:54.006771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:54.007558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:54.007558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:54.007558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:54.007558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:54.007939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:54.007939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:54.007939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:54.007939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:54.007939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:54.007939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:54.007939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:54.007939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:54.007939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:54.007939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:54.007939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:54.007939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:54.007939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:54.007939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:54.007939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:54.007939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:54.008803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:38:54.008803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:38:54.008803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:38:54.008803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:54.008803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:54.008803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:54.008803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:54.008803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:54.008803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:54.008803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:54.008803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:54.008803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:54.008803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:54.008803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:54.008803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:54.008803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:54.008803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:54.008803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:54.010213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:54.010213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:54.010213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:54.010213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:54.010213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:54.010213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:54.010213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:54.010213 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:54.010213 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:54.010213 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:54.010815 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:54.010815 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:54.010815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:54.010815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:54.010815 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:54.010815 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:54.010815 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:54.010815 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:54.010815 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:54.010815 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:54.010815 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:54.011614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:54.011614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:38:54.011614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:54.011614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:54.011614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:54.011614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:54.011614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:54.011614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:54.011614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:54.011614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:54.011614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:54.011614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:38:54.011614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:54.011614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:38:54.011614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:38:54.012460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:54.012460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:38:54.012460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:54.012460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:38:54.012460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:38:54.012460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:54.012460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:38:54.012460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:54.012460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:38:54.012460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:54.012460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:54.012460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:54.012460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:54.012460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:54.013381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:38:54.013381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:38:54.013381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:54.013381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:54.013381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:54.013381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:54.013381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:54.013381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:54.013381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:54.013381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:54.013381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:54.013381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:54.013381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:54.014294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:54.014294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:54.014294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:54.014294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:54.014294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:54.014294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:54.014294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:54.014294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:54.014294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:54.014294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:54.014294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:54.014294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:54.014983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:54.014983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:54.014983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:54.014983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:54.014983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:54.014983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:54.014983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:54.014983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:54.014983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:54.014983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:54.014983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:54.014983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:54.014983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:54.014983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:54.014983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:54.014983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:38:54.014983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:38:54.014983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:54.014983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:54.016050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:54.016050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:54.016050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:54.016050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:54.016050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:54.016050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:54.016050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:54.016050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:54.016050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:54.016050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:54.016050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:54.016050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:54.016050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:54.016050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:54.016050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:54.016050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:54.016050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:54.016050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:54.016050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:54.016050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:54.016739 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:54.016739 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:54.016739 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:54.016739 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:54.016739 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:54.016739 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:54.016968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:54.017190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:54.017190 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:54.017190 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:54.017190 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:54.017190 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:54.017190 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:54.017190 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:54.017190 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:54.018182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:54.018182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:54.018182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:54.018182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:54.018182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:54.018182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:38:54.018182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:54.018182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:54.018182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:54.018182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:54.018182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:54.018182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:54.018473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:54.018473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:54.018473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:54.018473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:54.018473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:54.018473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:54.018473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:54.018473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:54.018473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:54.018473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:54.018473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:54.018473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:54.018473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:54.018767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:54.018767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:54.019145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:54.019145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:54.019145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:54.019145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:54.019145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:54.019145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:54.019145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:54.019145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:54.019145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:54.019145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:54.019145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:54.019145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:54.019145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:54.019145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:54.019145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:54.019145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:54.019514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:38:54.019514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:54.019514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:54.019514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:54.019514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:54.019514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:54.019514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:54.019514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:54.019514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:54.019514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:54.019514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:54.019514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:54.019514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:54.019514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:54.019514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:54.019514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:54.019514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:54.019514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:54.019514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:54.019930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:54.019930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:54.019930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:58.731254 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:58.731727 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:58.731727 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:58.732434 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:58.761954 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:58.762369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:58.763227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:58.763227 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:58.763227 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:58.763227 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:58.763227 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:58.763227 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:58.763227 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:58.763227 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:58.763971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:58.763971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:58.763971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:58.763971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:58.763971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:58.763971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:38:58.763971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:58.763971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:58.763971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:58.763971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:58.763971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:58.763971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:58.764367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:58.764367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:58.764367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:58.764367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:58.764367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:58.764367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:58.764367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:58.764367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:58.764367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:58.764367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:58.764367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:58.764367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:58.764367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:58.764897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:58.764897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:58.764897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:58.764897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:58.764897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:58.764897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:58.764897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:58.764897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:58.764897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:58.764897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:58.764897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:58.764897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:58.764897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:58.764897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:58.764897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:58.765365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:58.765365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:58.765365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:58.765365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:38:58.765365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:58.765365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:58.765365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:58.765365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:58.765365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:58.765365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:58.765365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:58.765365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:58.765365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:58.765365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:58.765365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:58.765365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:58.765365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:58.765365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:58.765365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:58.765947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:58.765947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:58.765947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:58.765947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:58.765947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:58.765947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:58.962201 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:58.963848 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:58.964197 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:58.964974 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:58.965214 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:58.965405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:58.966067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:58.966067 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:58.966067 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:58.966067 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:58.966067 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:58.966067 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:58.966067 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:58.966067 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:58.967151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:58.967151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:58.967151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:58.967151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:58.967151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:58.967151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:38:58.967151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:58.967151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:58.967151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:58.967151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:58.967151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:58.967151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:58.967695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:58.967695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:58.967695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:58.967695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:58.967695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:58.967695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:58.967695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:58.967695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:58.967695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:58.967695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:58.967695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:58.967695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:58.967695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:58.968217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:58.968217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:58.968217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:58.968217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:58.968217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:58.968217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:58.968217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:58.968217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:58.968217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:58.968217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:58.968217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:58.968217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:58.968217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:58.968217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:58.968217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:58.968898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:58.968898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:58.968898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:58.968898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:38:58.968898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:58.968898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:58.968898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:58.968898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:58.968898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:58.968898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:58.968898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:58.968898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:58.968898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:58.968898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:58.968898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:58.968898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:58.968898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:58.968898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:58.968898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:58.969641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:58.969641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:58.969641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:58.969641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:58.969641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:58.969641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:59.000839 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:59.001201 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:59.001375 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:59.001612 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:38:59.002393 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:59.002617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:59.003430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:59.003430 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:59.003430 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:59.003430 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:59.003430 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:59.003430 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:59.003430 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:59.003430 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:59.004412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:59.004412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:38:59.004412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:59.004412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:59.004412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:59.004412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:59.004412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:59.004412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:59.004412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:59.004412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:59.004412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:59.004412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:38:59.004412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:59.004412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:38:59.004412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:38:59.005175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:59.005175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:38:59.005175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:59.005175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:38:59.005175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:38:59.005175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:59.005175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:38:59.005175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:59.005175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:38:59.005610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:59.005610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:59.005610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:59.005610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:59.005610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:59.005610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:38:59.005610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:38:59.005610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:38:59.005610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:59.005610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:59.005610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:59.005610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:59.006188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:59.006188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:59.006188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:59.006188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:59.006188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:59.006188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:59.006188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:59.006188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:59.006188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:59.006188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:59.006188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:59.006188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:59.006188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:59.006188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:59.006803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:59.006803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:59.006803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:59.006803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:59.006803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:59.006803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:59.006803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:59.006803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:59.006803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:59.006803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:59.006803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:59.006803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:59.006803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:59.006803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:59.007455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:59.007455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:59.007455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:59.007455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:59.007455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:59.007455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:59.007455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:38:59.007455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:38:59.007455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:38:59.007455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:59.007455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:59.007455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:59.007455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:59.007455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:59.007455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:59.007455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:59.007455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:59.007455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:59.007455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:59.008337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:59.008337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:59.008337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:59.008337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:59.008337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:59.008337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:59.008337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:59.008337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:59.008337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:59.008337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:59.008337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:59.008337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:59.009081 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:59.009308 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:59.009406 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:59.009541 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:59.009875 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:59.010185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:59.010673 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:59.010673 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:59.010673 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:59.010673 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:59.010673 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:59.010673 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:59.010673 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:59.010673 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:59.011539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:59.011539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:38:59.011539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:59.011539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:59.011539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:59.011539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:59.011539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:59.011539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:59.011539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:59.011539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:38:59.011539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:59.011539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:38:59.011539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:59.011539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:38:59.011539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:38:59.012001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:59.012001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:38:59.012001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:59.012001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:38:59.012001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:38:59.012001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:59.012001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:38:59.012001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:59.012001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:38:59.012404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:59.012404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:59.012404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:59.012404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:59.012404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:59.012404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:38:59.012404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:38:59.012404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:59.012404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:59.012404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:59.012404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:59.012773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:59.012773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:59.012773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:59.012773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:59.012773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:59.012773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:59.012773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:59.012773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:59.012773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:59.012773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:59.012773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:59.012773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:59.012773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:59.012773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:59.013173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:59.013173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:59.013173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:59.013173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:59.013944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:59.013944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:59.013944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:59.013944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:59.013944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:59.013944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:59.013944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:59.013944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:59.013944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:59.013944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:59.013944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:59.013944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:59.013944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:59.013944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:59.013944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:59.013944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:59.013944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:38:59.014462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:38:59.014462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:38:59.014462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:38:59.014462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:59.014462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:59.014462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:59.014462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:59.014462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:59.014462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:59.014462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:59.014462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:59.014462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:59.014462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:59.014462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:59.014462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:59.014462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:59.014462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:59.014462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:59.014462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:59.015026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:59.015026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:59.015026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:59.015026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:38:59.015346 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:38:59.015581 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:38:59.015776 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:38:59.015981 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:38:59.016283 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:59.016498 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:38:59.016643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:59.017187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:38:59.017187 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:59.017187 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:38:59.017187 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:38:59.017187 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:59.017187 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:38:59.017187 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:59.017187 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:38:59.018123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:59.018123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:38:59.018123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:59.018123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:38:59.018123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:59.018123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:38:59.018123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:59.018123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:59.018123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:59.018123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:59.018123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:59.018123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:59.018509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:38:59.018509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:59.018509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:38:59.018509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:59.018509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:38:59.018509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:38:59.018509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:59.018509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:38:59.018509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:59.018509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:38:59.018509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:38:59.018509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:59.018509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:38:59.018900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:59.018900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:38:59.019415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:59.019415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:38:59.019415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:38:59.019415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:38:59.019415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:38:59.019415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:38:59.019415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:59.019415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:38:59.019415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:38:59.019415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:59.019415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:38:59.019415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:38:59.019415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:38:59.019415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:38:59.019415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:38:59.019415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:38:59.019917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:38:59.019917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:38:59.019917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:59.019917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:59.019917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:38:59.019917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:38:59.019917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:38:59.019917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:38:59.019917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:38:59.019917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:38:59.019917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:38:59.019917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:38:59.019917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:38:59.019917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:38:59.019917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:38:59.019917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:38:59.019917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:38:59.019917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:38:59.019917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:38:59.020454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:38:59.020454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:38:59.020454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:03.731778 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:03.732360 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:03.732360 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:03.733256 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:03.764955 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:03.765346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:03.766202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:03.766202 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:03.766202 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:03.766202 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:03.766202 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:03.766202 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:03.766202 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:03.766202 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:03.767034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:03.767034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:03.767034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:03.767034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:03.767034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:03.767034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:39:03.767034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:03.767034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:03.767034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:03.767034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:03.767034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:03.767034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:03.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:03.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:03.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:03.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:03.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:03.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:03.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:03.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:03.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:03.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:03.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:03.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:03.767748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:03.768357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:03.768357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:03.768357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:03.768357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:03.768357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:03.768357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:03.768357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:03.768357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:03.768357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:03.768357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:03.768357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:03.768357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:03.768357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:03.768357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:03.768357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:03.769060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:03.769060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:03.769060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:03.769060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:39:03.769060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:03.769060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:03.769060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:03.769060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:03.769060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:03.769060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:03.769060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:03.769060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:03.769060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:03.769060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:03.769060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:03.769060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:03.769060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:03.769060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:03.769060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:03.769941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:03.769941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:03.769941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:03.769941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:03.769941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:03.769941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:03.963248 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:03.963628 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:03.963854 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:03.964829 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:03.965105 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:03.965336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:03.966143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:03.966143 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:03.966143 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:03.966143 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:03.966143 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:03.966143 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:03.966143 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:03.966143 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:03.967703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:03.967703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:03.967703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:03.967703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:03.967703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:03.967703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:39:03.967703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:03.967703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:03.967703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:03.967703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:03.967703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:03.967703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:03.968349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:03.968349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:03.968349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:03.968349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:03.968349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:03.968349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:03.968349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:03.968349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:03.968349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:03.968349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:03.968349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:03.968349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:03.968349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:03.968917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:03.968917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:03.969304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:03.969304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:03.969304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:03.969304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:03.969304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:03.969304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:03.969304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:03.969304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:03.969304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:03.969304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:03.969304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:03.969304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:03.969304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:03.969304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:03.969304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:03.969304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:03.970027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:39:03.970027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:03.970027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:03.970027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:03.970027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:03.970027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:03.970027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:03.970027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:03.970027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:03.970027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:03.970027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:03.970027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:03.970027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:03.970027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:03.970027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:03.970027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:03.970027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:03.970027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:03.970027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:03.970850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:03.970850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:03.970850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:04.001330 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:04.001551 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:04.001638 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:04.001746 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:39:04.002171 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:04.002378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:04.002888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:04.002888 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:04.002888 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:04.002888 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:04.002888 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:04.002888 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:04.002888 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:04.002888 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:04.003554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:04.003554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:39:04.003554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:04.003554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:04.003554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:04.003554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:04.003554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:04.003554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:04.003554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:04.003554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:04.003554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:04.003554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:39:04.003554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:04.003554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:39:04.003554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:39:04.003898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:04.003898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:39:04.003898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:04.003898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:39:04.003898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:39:04.003898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:04.003898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:39:04.003898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:04.003898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:39:04.004258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:04.004258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:04.004258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:04.004258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:04.004258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:04.004258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:39:04.004258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:39:04.004258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:39:04.004258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:04.004258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:04.004258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:04.004258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:04.004563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:04.004563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:04.004563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:04.004563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:04.004563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:04.004563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:04.004563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:04.004563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:04.004563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:04.004563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:04.004563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:04.004563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:04.004563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:04.004563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:04.004837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:04.004837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:04.004837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:04.004837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:04.005034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:04.005034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:04.005034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:04.005034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:04.005034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:04.005034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:04.005034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:04.005034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:04.005034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:04.005034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:04.005034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:04.005034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:04.005034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:04.005034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:04.005034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:04.005034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:04.005341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:39:04.005341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:39:04.005341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:39:04.005341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:04.005341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:04.005341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:04.005341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:04.005341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:04.005341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:04.005341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:04.005341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:04.005341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:04.005341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:04.005341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:04.005341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:04.005341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:04.005341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:04.005341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:04.005698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:04.005698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:04.005698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:04.005698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:04.005698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:04.005698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:04.005698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:04.009373 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:04.009572 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:04.009684 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:04.009843 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:04.010249 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:04.010586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:04.011345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:04.011345 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:04.011345 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:04.011345 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:04.011345 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:04.011345 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:04.011345 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:04.011345 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:04.012182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:04.012182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:39:04.012182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:04.012182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:04.012182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:04.012182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:04.012182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:04.012182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:04.012182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:04.012182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:04.012182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:04.012182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:39:04.012182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:04.012182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:39:04.012182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:39:04.012597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:04.012597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:39:04.012597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:04.012597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:39:04.012597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:39:04.012597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:04.012597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:39:04.012597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:04.012597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:39:04.013080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:04.013080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:04.013080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:04.013080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:04.013080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:04.013080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:39:04.013080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:39:04.013080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:04.013080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:04.013080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:04.013080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:04.013786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:04.013786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:04.013786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:04.013786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:04.013786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:04.013786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:04.013786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:04.013786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:04.013786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:04.013786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:04.013786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:04.013786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:04.013786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:04.013786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:04.014406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:04.014406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:04.014406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:04.014406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:04.014406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:04.014406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:04.014406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:04.014406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:04.014406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:04.014406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:04.014406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:04.014406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:04.014406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:04.014406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:04.015080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:04.015080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:04.015080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:04.015080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:04.015080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:04.015080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:04.015080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:39:04.015080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:39:04.015080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:04.015080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:04.015080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:04.015080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:04.015080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:04.015080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:04.015080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:04.015080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:04.015080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:04.015080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:04.015080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:04.015954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:04.015954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:04.015954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:04.015954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:04.015954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:04.015954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:04.015954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:04.015954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:04.015954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:04.015954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:04.015954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:04.015954 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:04.015954 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:04.016639 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:04.016639 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:04.016639 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:04.016832 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:04.017037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:04.017663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:04.017663 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:04.017663 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:04.017663 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:04.017663 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:04.017663 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:04.017663 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:04.017663 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:04.018553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:04.018553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:04.018553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:04.018553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:04.018553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:04.018553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:39:04.018553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:04.018553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:04.018553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:04.018553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:04.018553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:04.018553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:04.018921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:04.018921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:04.018921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:04.018921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:04.018921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:04.018921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:04.018921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:04.018921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:04.018921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:04.018921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:04.018921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:04.018921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:04.018921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:04.019267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:04.019267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:04.020026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:04.020026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:04.020026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:04.020026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:04.020026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:04.020026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:04.020026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:04.020026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:04.020026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:04.020026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:04.020026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:04.020026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:04.020026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:04.020026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:04.020026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:04.020026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:04.020484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:39:04.020484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:04.020484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:04.020484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:04.020484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:04.020484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:04.020484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:04.020484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:04.020484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:04.020484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:04.020484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:04.020484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:04.020484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:04.020484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:04.020484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:04.020484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:04.020484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:04.020484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:04.020484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:04.021029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:04.021029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:04.021029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:08.732270 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:08.733175 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:08.733175 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:08.733982 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:08.758344 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=242 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:39:08.768365 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:08.768778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:08.769597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:08.769597 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:08.769597 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:08.769597 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:08.769597 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:08.769597 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:08.769597 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:08.769597 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:08.770696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:08.770696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:08.770696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:08.770696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:08.770696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:08.770696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:39:08.770696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:08.770696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:08.770696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:08.770696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:08.770696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:08.770696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:08.771085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:08.771085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:08.771085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:08.771085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:08.771085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:08.771085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:08.771085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:08.771085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:08.771085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:08.771085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:08.771085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:08.771085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:08.771085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:08.771447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:08.771447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:08.771853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:08.771853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:08.771853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:08.771853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:08.771853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:08.771853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:08.771853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:08.771853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:08.771853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:08.771853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:08.771853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:08.771853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:08.771853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:08.771853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:08.771853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:08.771853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:08.772333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:39:08.772333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:08.772333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:08.772333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:08.772333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:08.772333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:08.772333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:08.772333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:08.772333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:08.772333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:08.772333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:08.772333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:08.772333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:08.772333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:08.772333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:08.772333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:08.772333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:08.772333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:08.772333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:08.772887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:08.772887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:08.772887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:08.963930 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:08.964301 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:08.964869 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:08.965990 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:08.966343 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:08.966560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:08.967387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:08.967387 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:08.967387 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:08.967387 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:08.967387 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:08.967387 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:08.967387 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:08.967387 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:08.968430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:08.968430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:08.968430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:08.968430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:08.968430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:08.968430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:39:08.968430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:08.968430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:08.968430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:08.968430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:08.968430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:08.968430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:08.969022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:08.969022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:08.969022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:08.969022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:08.969022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:08.969022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:08.969022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:08.969022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:08.969022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:08.969022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:08.969022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:08.969022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:08.969022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:08.969561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:08.969561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:08.969561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:08.969561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:08.969561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:08.969561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:08.969561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:08.969561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:08.969561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:08.969561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:08.969561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:08.969561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:08.969561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:08.969561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:08.969561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:08.970294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:08.970294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:08.970294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:08.970294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:39:08.970294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:08.970294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:08.970294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:08.970294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:08.970294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:08.970294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:08.970294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:08.970294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:08.970294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:08.970294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:08.970294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:08.970294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:08.970294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:08.970294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:08.970294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:08.971112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:08.971112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:08.971112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:08.971112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:08.971112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:08.971112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:09.001700 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:09.002137 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:09.002282 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:09.002392 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:39:09.002916 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:09.003182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:09.003880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:09.003880 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:09.003880 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:09.003880 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:09.003880 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:09.003880 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:09.003880 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:09.003880 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:09.004783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:09.004783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:39:09.004783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:09.004783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:09.004783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:09.004783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:09.004783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:09.004783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:09.004783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:09.004783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:09.004783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:09.004783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:39:09.004783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:09.004783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:39:09.004783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:39:09.005814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:09.005814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:39:09.005814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:09.005814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:39:09.005814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:39:09.005814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:09.005814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:39:09.005814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:09.005814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:39:09.006416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:09.006416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:09.006416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:09.006416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:09.006416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:09.006416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:39:09.006416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:39:09.006416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:39:09.006416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:09.006416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:09.006416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:09.006416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:09.007088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:09.007088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:09.007088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:09.007088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:09.007088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:09.007088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:09.007088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:09.007088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:09.007088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:09.007088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:09.007088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:09.007088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:09.007088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:09.007088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:09.007746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:09.007746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:09.007746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:09.007746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:09.007746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:09.007746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:09.007746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:09.007746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:09.007746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:09.007746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:09.007746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:09.007746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:09.007746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:09.007746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:09.008425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:09.008425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:09.008425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:09.008425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:09.008425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:09.008425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:09.008425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:39:09.008425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:39:09.008425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:39:09.008425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:09.008425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:09.008425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:09.008425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:09.008425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:09.008425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:09.008425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:09.008425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:09.008425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:09.008425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:09.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:09.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:09.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:09.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:09.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:09.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:09.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:09.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:09.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:09.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:09.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:09.009351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:09.010090 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:09.010474 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:09.010574 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:09.010730 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:09.011127 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:09.011459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:09.012207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:09.012207 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:09.012207 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:09.012207 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:09.012207 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:09.012207 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:09.012207 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:09.012207 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:09.012944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:09.012944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:39:09.012944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:09.012944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:09.012944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:09.012944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:09.012944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:09.012944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:09.012944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:09.012944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:09.012944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:09.012944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:39:09.012944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:09.012944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:39:09.012944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:39:09.013376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:09.013376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:39:09.013376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:09.013376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:39:09.013376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:39:09.013376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:09.013376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:39:09.013376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:09.013376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:39:09.013833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:09.013833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:09.013833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:09.013833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:09.013833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:09.013833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:39:09.013833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:39:09.013833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:09.013833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:09.013833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:09.013833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:09.014177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:09.014177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:09.014177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:09.014177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:09.014177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:09.014177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:09.014177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:09.014177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:09.014177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:09.014177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:09.014177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:09.014177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:09.014177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:09.014177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:09.014570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:09.014570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:09.014570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:09.014570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:09.014928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:09.014928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:09.014928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:09.014928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:09.014928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:09.014928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:09.014928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:09.014928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:09.014928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:09.014928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:09.014928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:09.014928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:09.014928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:09.014928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:09.014928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:09.014928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:09.014928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:39:09.015440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:39:09.015440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:09.015440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:09.015440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:09.015440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:09.015440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:09.015440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:09.015440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:09.015440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:09.015440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:09.015440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:09.015440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:09.015440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:09.015440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:09.015440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:09.015440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:09.015440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:09.015440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:09.015440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:09.015965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:09.015965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:09.015965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:09.015965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:09.015965 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:09.016187 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:09.016287 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:09.016438 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:09.016841 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:09.017160 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:09.017366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:09.017998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:09.017998 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:09.017998 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:09.017998 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:09.017998 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:09.017998 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:09.017998 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:09.017998 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:09.018845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:09.018845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:09.018845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:09.018845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:09.018845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:09.018845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:39:09.018845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:09.018845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:09.018845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:09.018845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:09.018845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:09.018845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:09.019209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:09.019209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:09.019209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:09.019209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:09.019209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:09.019209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:09.019209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:09.019209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:09.019209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:09.019209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:09.019209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:09.019209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:09.019209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:09.019553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:09.019553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:09.019809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:09.019809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:09.019809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:09.019809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:09.019809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:09.019809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:09.019809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:09.019809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:09.019809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:09.019809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:09.019809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:09.019809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:09.019809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:09.019809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:09.019809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:09.019809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:09.020255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:39:09.020255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:09.020255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:09.020255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:09.020255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:09.020255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:09.020255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:09.020255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:09.020255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:09.020255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:09.020255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:09.020255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:09.020255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:09.020255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:09.020255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:09.020255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:09.020255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:09.020255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:09.020255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:09.020912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:09.020912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:09.020912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:13.733057 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:13.734084 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:13.734084 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:13.735118 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:13.770517 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:13.770986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:13.771841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:13.771841 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:13.771841 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:13.771841 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:13.771841 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:13.771841 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:13.771841 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:13.771841 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:13.772582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:13.772582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:13.772582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:13.772582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:13.772582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:13.772582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:39:13.772582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:13.772582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:13.772582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:13.772582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:13.772582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:13.772582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:13.773240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:13.773240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:13.773240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:13.773240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:13.773240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:13.773240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:13.773240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:13.773240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:13.773240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:13.773240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:13.773240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:13.773240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:13.773240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:13.773895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:13.773895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:13.773895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:13.773895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:13.773895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:13.773895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:13.773895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:13.773895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:13.773895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:13.773895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:13.773895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:13.773895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:13.773895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:13.773895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:13.773895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:13.774630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:13.774630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:13.774630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:13.774630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:39:13.774630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:13.774630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:13.774630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:13.774630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:13.774630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:13.774630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:13.774630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:13.774630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:13.774630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:13.774630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:13.774630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:13.774630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:13.774630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:13.774630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:13.774630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:13.775558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:13.775558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:13.775558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:13.775558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:13.775558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:13.775558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:13.964665 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:13.966067 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:13.966178 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:13.967067 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:13.967435 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:13.967761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:13.968611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:13.968611 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:13.968611 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:13.968611 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:13.968611 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:13.968611 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:13.968611 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:13.968611 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:13.969563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:13.969563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:13.969563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:13.969563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:13.969563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:13.969563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:39:13.969563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:13.969563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:13.969563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:13.969563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:13.969563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:13.969563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:13.970302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:13.970302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:13.970302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:13.970302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:13.970302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:13.970302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:13.970302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:13.970302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:13.970302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:13.970302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:13.970302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:13.970302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:13.970302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:13.970929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:13.970929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:13.970929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:13.970929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:13.970929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:13.970929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:13.970929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:13.970929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:13.970929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:13.970929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:13.970929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:13.970929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:13.970929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:13.970929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:13.970929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:13.971576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:13.971576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:13.971576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:13.971576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:39:13.971576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:13.971576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:13.971576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:13.971576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:13.971576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:13.971576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:13.971576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:13.971576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:13.971576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:13.971576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:13.971576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:13.971576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:13.971576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:13.971576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:13.971576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:13.972569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:13.972569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:13.972569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:13.972569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:13.972569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:13.972569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:14.002440 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:14.002846 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:14.003027 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:14.003210 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:39:14.003821 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:14.004152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:14.004883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:14.004883 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:14.004883 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:14.004883 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:14.004883 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:14.004883 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:14.004883 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:14.004883 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:14.006069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:14.006069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:39:14.006069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:14.006069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:14.006069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:14.006069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:14.006069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:14.006069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:14.006069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:14.006069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:14.006069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:14.006069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:39:14.006069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:14.006069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:39:14.006069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:39:14.006547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:14.006547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:39:14.006547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:14.006547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:39:14.006547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:39:14.006547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:14.006547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:39:14.006547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:14.006547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:39:14.007373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:14.007373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:14.007373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:14.007373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:14.007373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:14.007373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:39:14.007373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:39:14.007373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:39:14.007373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:14.007373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:14.007373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:14.007373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:14.007813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:14.007813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:14.007813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:14.007813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:14.007813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:14.007813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:14.007813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:14.007813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:14.007813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:14.007813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:14.007813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:14.007813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:14.007813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:14.007813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:14.008169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:14.008169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:14.008169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:14.008169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:14.009046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:14.009046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:14.009046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:14.009046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:14.009046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:14.009046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:14.009046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:14.009046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:14.009046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:14.009046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:14.009046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:14.009046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:14.009046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:14.009046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:14.009046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:14.009046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:14.009525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:39:14.009525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:39:14.009525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:39:14.009525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:14.009525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:14.009525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:14.009525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:14.009525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:14.009525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:14.009525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:14.009525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:14.009525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:14.009525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:14.009525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:14.009525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:14.009525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:14.009525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:14.009525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:14.010089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:14.010089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:14.010089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:14.010089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:14.010089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:14.010089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:14.010089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:14.010551 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:14.010829 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:14.010942 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:14.011024 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:14.011466 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:14.011867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:14.012608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:14.012608 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:14.012608 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:14.012608 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:14.012608 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:14.012608 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:14.012608 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:14.012608 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:14.013344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:14.013344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:39:14.013344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:14.013344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:14.013344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:14.013344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:14.013344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:14.013344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:14.013344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:14.013344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:14.013344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:14.013344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:39:14.013344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:14.013344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:39:14.013344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:39:14.014253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:14.014253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:39:14.014253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:14.014253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:39:14.014253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:39:14.014253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:14.014253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:39:14.014253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:14.014253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:39:14.014253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:14.014253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:14.014253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:14.014253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:14.014253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:14.014975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:39:14.014975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:39:14.014975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:14.014975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:14.014975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:14.014975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:14.014975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:14.014975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:14.014975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:14.014975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:14.014975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:14.014975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:14.014975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:14.015774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:14.015774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:14.015774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:14.015774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:14.015774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:14.015774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:14.015774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:14.015774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:14.015774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:14.015774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:14.015774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:14.015774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:14.016444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:14.016444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:14.016444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:14.016444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:14.016444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:14.016444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:14.016444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:14.016444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:14.016444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:14.016444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:14.016444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:14.016444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:14.016444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:14.016444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:14.016444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:14.016444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:39:14.016444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:39:14.016444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:14.016444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:14.017495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:14.017495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:14.017495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:14.017495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:14.017495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:14.017495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:14.017495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:14.017495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:14.017495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:14.017495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:14.017495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:14.017495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:14.017495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:14.017495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:14.017495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:14.017495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:14.017495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:14.017495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:14.017495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:14.017495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:14.018116 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:14.018116 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:14.018116 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:14.018116 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:14.018116 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:14.018116 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:14.018418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:14.018418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:14.018418 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:14.018418 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:14.018418 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:14.018418 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:14.018418 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:14.018418 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:14.018418 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:14.019228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:14.019228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:14.019228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:14.019228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:14.019228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:14.019228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:39:14.019228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:14.019228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:14.019228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:14.019228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:14.019228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:14.019228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:14.019620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:14.019620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:14.019620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:14.019620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:14.019620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:14.019620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:14.019620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:14.019620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:14.019620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:14.019620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:14.019620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:14.019620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:14.019620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:14.020011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:14.020011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:14.020204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:14.020204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:14.020204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:14.020204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:14.020204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:14.020204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:14.020204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:14.020204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:14.020204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:14.020204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:14.020204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:14.020204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:14.020204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:14.020204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:14.020204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:14.020204 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:14.020809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:39:14.020809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:14.020809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:14.020809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:14.020809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:14.020809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:14.020809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:14.020809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:14.020809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:14.020809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:14.020809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:14.020809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:14.020809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:14.020809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:14.020809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:14.020809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:14.020809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:14.020809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:14.020809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:14.021457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:14.021457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:14.021457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:18.733821 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:18.734723 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:18.734723 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:18.736219 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:18.768668 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:18.769076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:18.769995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:18.769995 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:18.769995 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:18.769995 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:18.769995 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:18.769995 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:18.769995 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:18.769995 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:18.770885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:18.770885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:18.770885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:18.770885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:18.770885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:18.770885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:39:18.770885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:18.770885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:18.770885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:18.770885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:18.770885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:18.770885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:18.771481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:18.771481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:18.771481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:18.771481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:18.771481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:18.771481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:18.771481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:18.771481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:18.771481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:18.771481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:18.771481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:18.771481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:18.771481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:18.772103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:18.772103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:18.772103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:18.772103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:18.772103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:18.772103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:18.772103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:18.772103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:18.772103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:18.772103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:18.772103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:18.772103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:18.772103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:18.772103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:18.772103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:18.772862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:18.772862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:18.772862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:18.772862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:39:18.772862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:18.772862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:18.772862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:18.772862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:18.772862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:18.772862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:18.772862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:18.772862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:18.772862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:18.772862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:18.772862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:18.772862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:18.772862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:18.772862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:18.772862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:18.773716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:18.773716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:18.773716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:18.773716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:18.773716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:18.773716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:18.967527 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:18.968097 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:18.968325 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:18.969313 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:18.969539 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:18.969667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:18.970356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:18.970356 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:18.970356 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:18.970356 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:18.970356 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:18.970356 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:18.970356 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:18.970356 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:18.971962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:18.971962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:18.971962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:18.971962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:18.971962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:18.971962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:39:18.971962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:18.971962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:18.971962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:18.971962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:18.971962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:18.971962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:18.972758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:18.972758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:18.972758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:18.972758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:18.972758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:18.972758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:18.972758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:18.972758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:18.972758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:18.972758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:18.972758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:18.972758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:18.972758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:18.973592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:18.973592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:18.973592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:18.973592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:18.973592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:18.973592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:18.973592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:18.973592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:18.973592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:18.973592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:18.973592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:18.973592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:18.973592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:18.973592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:18.973592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:18.974491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:18.974491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:18.974491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:18.974491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:39:18.974491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:18.974491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:18.974491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:18.974491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:18.974491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:18.974491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:18.974491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:18.974491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:18.974491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:18.974491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:18.974491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:18.974491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:18.974491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:18.974491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:18.974491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:18.975535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:18.975535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:18.975535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:18.975535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:18.975535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:18.975535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:19.003018 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:19.003238 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:19.003343 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:19.003451 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:39:19.003842 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:19.004029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:19.004485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:19.004485 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:19.004485 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:19.004485 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:19.004485 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:19.004485 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:19.004485 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:19.004485 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:19.005125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:19.005125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:39:19.005125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:19.005125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:19.005125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:19.005125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:19.005125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:19.005125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:19.005125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:19.005125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:19.005125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:19.005125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:39:19.005125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:19.005125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:39:19.005125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:39:19.006013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:19.006013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:39:19.006013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:19.006013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:39:19.006013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:39:19.006013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:19.006013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:39:19.006013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:19.006013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:39:19.006013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:19.006013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:19.006013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:19.006013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:19.006013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:19.006836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:39:19.006836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:39:19.006836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:39:19.006836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:19.006836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:19.006836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:19.006836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:19.006836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:19.006836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:19.006836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:19.006836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:19.006836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:19.006836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:19.007655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:19.007655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:19.007655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:19.007655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:19.007655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:19.007655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:19.007655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:19.007655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:19.007655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:19.007655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:19.007655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:19.007655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:19.008232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:19.008232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:19.008232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:19.008232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:19.008232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:19.008232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:19.008232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:19.008232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:19.008232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:19.008232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:19.008232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:19.008232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:19.008232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:19.008232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:19.008232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:19.008232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:19.008854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:39:19.008854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:39:19.008854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:39:19.008854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:19.008854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:19.008854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:19.008854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:19.008854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:19.008854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:19.008854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:19.008854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:19.008854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:19.008854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:19.008854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:19.008854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:19.008854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:19.008854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:19.008854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:19.009347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:19.009347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:19.009347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:19.009347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:19.009347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:19.009347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:19.009347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:19.011049 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:19.011385 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:19.011659 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:19.011951 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:19.012298 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:19.012562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:19.013347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:19.013347 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:19.013347 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:19.013347 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:19.013347 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:19.013347 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:19.013347 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:19.013347 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:19.014590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:19.014590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:39:19.014590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:19.014590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:19.014590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:19.014590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:19.014590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:19.014590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:19.014590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:19.014590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:19.014590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:19.014590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:39:19.014590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:19.014590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:39:19.014590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:39:19.015333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:19.015333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:39:19.015333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:19.015333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:39:19.015333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:39:19.015333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:19.015333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:39:19.015333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:19.015333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:39:19.015938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:19.015938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:19.015938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:19.015938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:19.015938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:19.015938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:39:19.015938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:39:19.015938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:19.015938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:19.015938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:19.015938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:19.016466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:19.016466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:19.016466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:19.016466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:19.016466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:19.016466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:19.016466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:19.016466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:19.016466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:19.016466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:19.016466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:19.016466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:19.016466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:19.016466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:19.016865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:19.016865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:19.016865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:19.016865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:19.017851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:19.017851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:19.017851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:19.017851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:19.017851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:19.017851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:19.017851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:19.017851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:19.017851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:19.017851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:19.017851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:19.017851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:19.017851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:19.017851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:19.017851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:19.017851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:19.017851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:39:19.018378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:39:19.018378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:19.018378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:19.018378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:19.018378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:19.018378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:19.018378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:19.018378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:19.018378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:19.018378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:19.018378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:19.018378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:19.018378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:19.018378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:19.018378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:19.018378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:19.018378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:19.018378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:19.018378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:19.018927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:19.018927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:19.018927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:19.018927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:19.018927 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:19.018927 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:19.018927 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:19.018927 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:19.019458 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:19.019774 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:19.019992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:19.020690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:19.020690 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:19.020690 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:19.020690 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:19.020690 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:19.020690 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:19.020690 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:19.020690 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:19.021572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:19.021572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:19.021572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:19.021572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:19.021572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:19.021572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:39:19.021572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:19.021572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:19.021572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:19.021572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:19.021572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:19.021572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:19.021947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:19.021947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:19.021947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:19.021947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:19.021947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:19.021947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:19.021947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:19.021947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:19.021947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:19.021947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:19.021947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:19.021947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:19.021947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:19.022304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:19.022304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:19.022691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:19.022691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:19.022691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:19.022691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:19.022691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:19.022691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:19.022691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:19.022691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:19.022691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:19.022691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:19.022691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:19.022691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:19.022691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:19.022691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:19.022691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:19.022691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:19.023146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:39:19.023146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:19.023146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:19.023146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:19.023146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:19.023146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:19.023146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:19.023146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:19.023146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:19.023146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:19.023146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:19.023146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:19.023146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:19.023146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:19.023146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:19.023146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:19.023146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:19.023146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:19.023146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:19.023716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:19.023716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:19.023716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:23.734897 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:23.735544 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:23.735544 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:23.736995 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:23.774034 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:23.774913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:23.775965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:23.775965 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:23.775965 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:23.775965 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:23.775965 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:23.775965 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:23.775965 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:23.775965 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:23.777154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:23.777154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:23.777154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:23.777154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:23.777154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:23.777154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:39:23.777154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:23.777154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:23.777154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:23.777154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:23.777154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:23.777154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:23.778014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:23.778014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:23.778014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:23.778014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:23.778014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:23.778014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:23.778014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:23.778014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:23.778014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:23.778014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:23.778014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:23.778014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:23.778014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:23.778669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:23.778669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:23.778669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:23.778669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:23.778669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:23.778669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:23.778669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:23.778669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:23.778669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:23.778669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:23.778669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:23.778669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:23.778669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:23.778669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:23.778669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:23.779516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:23.779516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:23.779516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:23.779516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:39:23.779516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:23.779516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:23.779516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:23.779516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:23.779516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:23.779516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:23.779516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:23.779516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:23.779516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:23.779516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:23.779516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:23.779516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:23.779516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:23.779516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:23.779516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:23.780446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:23.780446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:23.780446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:23.780446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:23.780446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:23.780446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:23.968332 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:23.968446 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:23.968550 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:23.969373 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:23.969562 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:23.969789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:23.970337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:23.970337 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:23.970337 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:23.970337 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:23.970337 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:23.970337 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:23.970337 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:23.970337 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:23.971236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:23.971236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:23.971236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:23.971236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:23.971236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:23.971236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:39:23.971236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:23.971236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:23.971236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:23.971236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:23.971236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:23.971236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:23.971586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:23.971586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:23.971586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:23.971586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:23.971586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:23.971586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:23.971586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:23.971586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:23.971586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:23.971586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:23.971586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:23.971586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:23.971586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:23.971957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:23.971957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:23.972063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:23.972063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:23.972063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:23.972063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:23.972063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:23.972063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:23.972063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:23.972063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:23.972063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:23.972063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:23.972063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:23.972063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:23.972063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:23.972063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:23.972063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:23.972063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:23.972597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:39:23.972597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:23.972597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:23.972597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:23.972597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:23.972597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:23.972597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:23.972597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:23.972597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:23.972597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:23.972597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:23.972597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:23.972597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:23.972597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:23.972597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:23.972597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:23.972597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:23.972597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:23.972597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:23.973101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:23.973101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:23.973101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:24.003429 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:24.004523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:24.004523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:24.004523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:39:24.004957 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:24.005278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:24.006105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:24.006105 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:24.006105 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:24.006105 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:24.006105 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:24.006105 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:24.006105 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:24.006105 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:24.008391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:24.008391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:39:24.008391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:24.008391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:24.008391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:24.008391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:24.008391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:24.008391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:24.008391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:24.008391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:24.008391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:24.008391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:39:24.008391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:24.008391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:39:24.008391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:39:24.008859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:24.008859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:39:24.008859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:24.008859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:39:24.008859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:39:24.008859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:24.008859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:39:24.008859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:24.008859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:39:24.008859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:24.008859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:24.008859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:24.008859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:24.008859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:24.009259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:39:24.009259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:39:24.009259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:39:24.009259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:24.009259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:24.009259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:24.009259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:24.009259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:24.009259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:24.009259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:24.009259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:24.009259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:24.009259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:24.009614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:24.009614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:24.009614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:24.009614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:24.009614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:24.009614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:24.009614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:24.009614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:24.009614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:24.009614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:24.009614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:24.009614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:24.009949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:24.009949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:24.009949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:24.009949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:24.009949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:24.009949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:24.009949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:24.009949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:24.009949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:24.009949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:24.009949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:24.009949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:24.009949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:24.009949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:24.009949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:24.009949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:24.010422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:39:24.010422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:39:24.010422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:39:24.010422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:24.010422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:24.010422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:24.010422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:24.010422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:24.010422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:24.010422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:24.010422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:24.010422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:24.010422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:24.010422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:24.010422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:24.010422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:24.010422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:24.010422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:24.010914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:24.010914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:24.010914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:24.010914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:24.010914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:24.010914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:24.010914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:24.012030 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:24.012030 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:24.012030 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:24.012030 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:24.012192 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:24.012362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:24.012854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:24.012854 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:24.012854 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:24.012854 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:24.012854 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:24.012854 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:24.012854 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:24.012854 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:24.013580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:24.013580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:39:24.013580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:24.013580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:24.013580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:24.013580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:24.013580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:24.013580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:24.013580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:24.013580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:24.013580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:24.013580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:39:24.013580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:24.013580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:39:24.013580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:39:24.014000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:24.014000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:39:24.014000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:24.014000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:39:24.014000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:39:24.014000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:24.014000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:39:24.014000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:24.014000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:39:24.014647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:24.014647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:24.014647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:24.014647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:24.014647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:24.014647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:39:24.014647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:39:24.014647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:24.014647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:24.014647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:24.014647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:24.015015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:24.015015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:24.015015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:24.015015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:24.015015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:24.015015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:24.015015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:24.015015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:24.015015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:24.015015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:24.015015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:24.015015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:24.015015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:24.015015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:24.015404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:24.015404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:24.015404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:24.015404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:24.016102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:24.016102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:24.016102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:24.016102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:24.016102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:24.016102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:24.016102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:24.016102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:24.016102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:24.016102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:24.016102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:24.016102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:24.016102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:24.016102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:24.016102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:24.016102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:24.016102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:39:24.016623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:39:24.016623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:24.016623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:24.016623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:24.016623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:24.016623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:24.016623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:24.016623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:24.016623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:24.016623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:24.016623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:24.016623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:24.016623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:24.016623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:24.016623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:24.016623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:24.016623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:24.016623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:24.016623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:24.018300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:24.018300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:24.018300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:24.018300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:24.018476 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:24.018758 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:24.018871 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:24.019058 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:24.019479 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:24.019820 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:24.020055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:24.020694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:24.020694 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:24.020694 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:24.020694 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:24.020694 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:24.020694 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:24.020694 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:24.020694 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:24.021506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:24.021506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:24.021506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:24.021506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:24.021506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:24.021506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:39:24.021506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:24.021506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:24.021506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:24.021506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:24.021506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:24.021506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:24.021885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:24.021885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:24.021885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:24.021885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:24.021885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:24.021885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:24.021885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:24.021885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:24.021885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:24.021885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:24.021885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:24.021885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:24.021885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:24.022270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:24.022270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:24.022656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:24.022656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:24.022656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:24.022656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:24.022656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:24.022656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:24.022656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:24.022656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:24.022656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:24.022656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:24.022656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:24.022656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:24.022656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:24.022656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:24.022656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:24.022656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:24.023356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:39:24.023356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:24.023356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:24.023356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:24.023356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:24.023356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:24.023356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:24.023356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:24.023356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:24.023356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:24.023356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:24.023356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:24.023356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:24.023356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:24.023356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:24.023356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:24.023356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:24.023356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:24.023356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:24.024091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:24.024091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:24.024091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:28.735338 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:28.736239 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:28.736239 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:28.737570 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:28.773264 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:28.773754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:28.774205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:28.774205 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:28.774205 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:28.774205 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:28.774205 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:28.774205 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:28.774205 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:28.774205 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:28.775332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:28.775332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:28.775332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:28.775332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:28.775332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:28.775332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:39:28.775332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:28.775332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:28.775332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:28.775332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:28.775332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:28.775332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:28.776034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:28.776034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:28.776034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:28.776034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:28.776034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:28.776034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:28.776034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:28.776034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:28.776034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:28.776034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:28.776034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:28.776034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:28.776034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:28.776829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:28.776829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:28.776829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:28.776829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:28.776829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:28.776829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:28.776829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:28.776829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:28.776829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:28.776829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:28.776829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:28.776829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:28.776829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:28.776829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:28.776829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:28.778106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:28.778106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:28.778106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:28.778106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:39:28.778106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:28.778106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:28.778106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:28.778106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:28.778106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:28.778106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:28.778106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:28.778106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:28.778106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:28.778106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:28.778106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:28.778106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:28.778106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:28.778106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:28.778106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:28.779501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:28.779501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:28.779501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:28.779501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:28.779501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:28.779501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:28.968612 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:28.969302 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:28.969560 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:28.970560 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:28.970830 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:28.970946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:28.971655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:28.971655 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:28.971655 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:28.971655 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:28.971655 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:28.971655 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:28.971655 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:28.971655 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:28.973468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:28.973468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:28.973468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:28.973468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:28.973468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:28.973468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:39:28.973468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:28.973468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:28.973468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:28.973468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:28.973468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:28.973468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:28.974127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:28.974127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:28.974127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:28.974127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:28.974127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:28.974127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:28.974127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:28.974127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:28.974127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:28.974127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:28.974127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:28.974127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:28.974127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:28.974765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:28.974765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:28.974765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:28.974765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:28.974765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:28.974765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:28.974765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:28.974765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:28.974765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:28.974765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:28.974765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:28.974765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:28.974765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:28.974765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:28.974765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:28.975430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:28.975430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:28.975430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:28.975430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:39:28.975430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:28.975430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:28.975430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:28.975430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:28.975430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:28.975430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:28.975430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:28.975430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:28.975430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:28.975430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:28.975430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:28.975430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:28.975430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:28.975430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:28.975430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:28.976342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:28.976342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:28.976342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:28.976342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:28.976342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:28.976342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:29.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:29.004441 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:29.004652 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:29.004993 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:39:29.007929 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:29.008184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:29.009118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:29.009118 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:29.009118 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:29.009118 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:29.009118 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:29.009118 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:29.009118 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:29.009118 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:29.011243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:29.011243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:39:29.011243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:29.011243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:29.011243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:29.011243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:29.011243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:29.011243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:29.011243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:29.011243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:29.011243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:29.011243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:39:29.011243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:29.011243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:39:29.011243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:39:29.012217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:29.012217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:39:29.012217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:29.012217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:39:29.012217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:39:29.012217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:29.012217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:39:29.012217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:29.012217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:39:29.012925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:29.012925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:29.012925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:29.012925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:29.012925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:29.012925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:39:29.012925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:39:29.012925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:39:29.012925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:29.012925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:29.012925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:29.012925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:29.013640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:29.013640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:29.013640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:29.013640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:29.013640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:29.013640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:29.013640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:29.013640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:29.013640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:29.013640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:29.013640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:29.013640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:29.013640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:29.013640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:29.014151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:29.014151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:29.014151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:29.014151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:29.014151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:29.014151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:29.014151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:29.014151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:29.014151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:29.014151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:29.014151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:29.014151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:29.014151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:29.014151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:29.014563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:29.014563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:29.014563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:29.014563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:29.014563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:29.014563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:29.014563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:39:29.014563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:39:29.014563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:39:29.014563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:29.014563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:29.014563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:29.014563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:29.014563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:29.014563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:29.014563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:29.014563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:29.014563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:29.014563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:29.015054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:29.015054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:29.015054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:29.015054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:29.015054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:29.015054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:29.015054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:29.015054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:29.015054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:29.015054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:29.015054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:29.015054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:29.015054 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:29.015054 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:29.015389 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:29.015389 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:29.015389 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:29.015389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:29.015790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:29.015790 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:29.015790 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:29.015790 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:29.015790 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:29.015790 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:29.015790 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:29.015790 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:29.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:29.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:39:29.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:29.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:29.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:29.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:29.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:29.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:29.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:29.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:29.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:29.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:39:29.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:29.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:39:29.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:39:29.016935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:29.016935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:39:29.016935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:29.016935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:39:29.016935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:39:29.016935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:29.016935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:39:29.016935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:29.016935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:39:29.017557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:29.017557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:29.017557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:29.017557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:29.017557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:29.017557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:39:29.017557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:39:29.017557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:29.017557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:29.017557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:29.017557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:29.017827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:29.017827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:29.017827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:29.017827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:29.017827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:29.017827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:29.017827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:29.017827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:29.017827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:29.017827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:29.017827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:29.017827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:29.017827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:29.017827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:29.018106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:29.018106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:29.018106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:29.018106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:29.018373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:29.018373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:29.018373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:29.018373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:29.018373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:29.018373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:29.018373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:29.018373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:29.018373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:29.018373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:29.018373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:29.018373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:29.018373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:29.018373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:29.018373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:29.018373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:29.018373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:39:29.018753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:39:29.018753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:29.018753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:29.018753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:29.018753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:29.018753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:29.018753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:29.018753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:29.018753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:29.018753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:29.018753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:29.018753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:29.018753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:29.018753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:29.018753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:29.018753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:29.018753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:29.018753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:29.018753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:29.019178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:29.019178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:29.019178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:29.019178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:29.019178 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:29.019178 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:29.019178 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:29.019178 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:29.019386 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:29.019613 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:29.019807 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:29.020289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:29.020289 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:29.020289 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:29.020289 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:29.020289 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:29.020289 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:29.020289 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:29.020289 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:29.020953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:29.020953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:29.020953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:29.020953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:29.020953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:29.020953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:39:29.020953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:29.020953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:29.020953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:29.020953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:29.020953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:29.020953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:29.021227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:29.021227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:29.021227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:29.021227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:29.021227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:29.021227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:29.021227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:29.021227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:29.021227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:29.021227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:29.021227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:29.021227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:29.021227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:29.021565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:29.021565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:29.021614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:29.021614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:29.021614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:29.021614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:29.021614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:29.021614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:29.021614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:29.021614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:29.021614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:29.021614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:29.021614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:29.021614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:29.021614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:29.021614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:29.021614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:29.021614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:29.021975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:39:29.021975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:29.021975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:29.021975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:29.021975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:29.021975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:29.021975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:29.021975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:29.021975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:29.021975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:29.021975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:29.021975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:29.021975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:29.021975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:29.021975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:29.021975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:29.021975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:29.021975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:29.021975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:29.022354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:29.022354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:29.022354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:33.735857 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:33.736477 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:33.736710 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:33.738076 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:33.773025 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:33.773529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:33.774247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:33.774247 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:33.774247 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:33.774247 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:33.774247 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:33.774247 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:33.774247 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:33.774247 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:33.775091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:33.775091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:33.775091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:33.775091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:33.775091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:33.775091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:39:33.775091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:33.775091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:33.775091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:33.775091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:33.775091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:33.775091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:33.775777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:33.775777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:33.775777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:33.775777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:33.775777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:33.775777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:33.775777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:33.775777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:33.775777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:33.775777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:33.775777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:33.775777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:33.775777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:33.776418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:33.776418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:33.776418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:33.776418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:33.776418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:33.776418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:33.776418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:33.776418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:33.776418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:33.776418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:33.776418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:33.776418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:33.776418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:33.776418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:33.776418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:33.777213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:33.777213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:33.777213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:33.777213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:39:33.777213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:33.777213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:33.777213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:33.777213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:33.777213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:33.777213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:33.777213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:33.777213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:33.777213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:33.777213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:33.777213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:33.777213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:33.777213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:33.777213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:33.777213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:33.778175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:33.778175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:33.778175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:33.778175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:33.778175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:33.778175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:33.970273 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:33.971072 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:33.971292 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:33.972879 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:33.973482 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:33.973767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:33.974738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:33.974738 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:33.974738 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:33.974738 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:33.974738 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:33.974738 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:33.974738 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:33.974738 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:33.976518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:33.976518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:33.976518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:33.976518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:33.976518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:33.976518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:39:33.976518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:33.976518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:33.976518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:33.976518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:33.976518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:33.976518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:33.977160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:33.977160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:33.977160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:33.977160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:33.977160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:33.977160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:33.977160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:33.977160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:33.977160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:33.977160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:33.977160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:33.977160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:33.977160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:33.977773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:33.977773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:33.978568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:33.978568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:33.978568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:33.978568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:33.978568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:33.978568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:33.978568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:33.978568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:33.978568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:33.978568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:33.978568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:33.978568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:33.978568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:33.978568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:33.978568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:33.978568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:33.979766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:39:33.979766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:33.979766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:33.979766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:33.979766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:33.979766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:33.979766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:33.979766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:33.979766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:33.979766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:33.979766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:33.979766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:33.979766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:33.979766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:33.979766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:33.979766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:33.979766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:33.979766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:33.979766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:33.980704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:33.980704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:33.980704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:34.004514 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:34.005120 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:34.005293 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:34.005482 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:39:34.006271 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:34.006639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:34.007625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:34.007625 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:34.007625 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:34.007625 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:34.007625 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:34.007625 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:34.007625 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:34.007625 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:34.009018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:34.009018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:39:34.009018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:34.009018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:34.009018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:34.009018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:34.009018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:34.009018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:34.009018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:34.009018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:34.009018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:34.009018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:39:34.009018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:34.009018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:39:34.009018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:39:34.009765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:34.009765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:39:34.009765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:34.009765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:39:34.009765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:39:34.009765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:34.009765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:39:34.009765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:34.009765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:39:34.010346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:34.010346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:34.010346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:34.010346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:34.010346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:34.010346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:39:34.010346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:39:34.010346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:39:34.010346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:34.010346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:34.010346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:34.010346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:34.010995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:34.010995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:34.010995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:34.010995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:34.010995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:34.010995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:34.010995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:34.010995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:34.010995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:34.010995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:34.010995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:34.010995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:34.010995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:34.010995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:34.011637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:34.011637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:34.011637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:34.011637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:34.012360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:34.012360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:34.012360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:34.012360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:34.012360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:34.012360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:34.012360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:34.012360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:34.012360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:34.012360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:34.012360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:34.012360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:34.012360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:34.012360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:34.012360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:34.012360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:34.013182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:39:34.013182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:39:34.013182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:39:34.013182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:34.013182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:34.013182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:34.013182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:34.013182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:34.013182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:34.013182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:34.013182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:34.013182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:34.013182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:34.013182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:34.013182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:34.013182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:34.013182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:34.013182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:34.014180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:34.014180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:34.014180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:34.014180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:34.014180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:34.014180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:34.014180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:34.014666 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:34.014937 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:34.015026 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:34.015166 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:34.015621 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:34.015904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:34.016684 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:34.016684 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:34.016684 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:34.016684 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:34.016684 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:34.016684 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:34.016684 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:34.016684 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:34.017494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:34.017494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:39:34.017494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:34.017494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:34.017494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:34.017494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:34.017494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:34.017494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:34.017494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:34.017494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:34.017494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:34.017494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:39:34.017494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:34.017494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:39:34.017494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:39:34.017931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:34.017931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:39:34.017931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:34.017931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:39:34.017931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:39:34.017931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:34.017931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:39:34.017931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:34.017931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:39:34.018407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:34.018407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:34.018407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:34.018407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:34.018407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:34.018407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:39:34.018407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:39:34.018407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:34.018407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:34.018407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:34.018407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:34.018773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:34.018773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:34.018773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:34.018773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:34.018773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:34.018773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:34.018773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:34.018773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:34.018773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:34.018773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:34.018773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:34.018773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:34.018773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:34.018773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:34.019146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:34.019146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:34.019146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:34.019146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:34.019619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:34.019619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:34.019619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:34.019619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:34.019619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:34.019619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:34.019619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:34.019619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:34.019619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:34.019619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:34.019619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:34.019619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:34.019619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:34.019619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:34.019619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:34.019619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:34.019619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:39:34.020233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:39:34.020233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:34.020233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:34.020233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:34.020233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:34.020233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:34.020233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:34.020233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:34.020233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:34.020233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:34.020233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:34.020233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:34.020233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:34.020233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:34.020233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:34.020233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:34.020233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:34.020233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:34.020233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:34.020815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:34.020815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:34.020815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:34.020815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:34.020815 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:34.020815 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:34.020815 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:34.020815 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:34.021070 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:34.021304 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:34.021563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:34.022338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:34.022338 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:34.022338 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:34.022338 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:34.022338 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:34.022338 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:34.022338 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:34.022338 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:34.023319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:34.023319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:34.023319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:34.023319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:34.023319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:34.023319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:39:34.023319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:34.023319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:34.023319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:34.023319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:34.023319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:34.023319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:34.023730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:34.023730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:34.023730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:34.023730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:34.023730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:34.023730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:34.023730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:34.023730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:34.023730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:34.023730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:34.023730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:34.023730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:34.023730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:34.024147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:34.024147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:34.024347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:34.024347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:34.024347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:34.024347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:34.024347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:34.024347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:34.024347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:34.024347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:34.024347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:34.024347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:34.024347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:34.024347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:34.024347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:34.024347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:34.024347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:34.024347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:34.024844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:39:34.024844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:34.024844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:34.024844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:34.024844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:34.024844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:34.024844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:34.024844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:34.024844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:34.024844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:34.024844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:34.024844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:34.024844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:34.024844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:34.024844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:34.024844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:34.024844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:34.024844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:34.024844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:34.025378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:34.025378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:34.025378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:38.736474 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:38.737184 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:38.737184 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:38.738336 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:38.776564 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:38.777013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:38.777961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:38.777961 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:38.777961 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:38.777961 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:38.777961 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:38.777961 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:38.777961 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:38.777961 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:38.778748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:38.778748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:38.778748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:38.778748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:38.778748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:38.778748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:39:38.778748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:38.778748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:38.778748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:38.778748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:38.778748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:38.778748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:38.779487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:38.779487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:38.779487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:38.779487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:38.779487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:38.779487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:38.779487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:38.779487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:38.779487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:38.779487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:38.779487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:38.779487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:38.779487 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:38.780219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:38.780219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:38.780219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:38.780219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:38.780219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:38.780219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:38.780219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:38.780219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:38.780219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:38.780219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:38.780219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:38.780219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:38.780219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:38.780219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:38.780219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:38.781068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:38.781068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:38.781068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:38.781068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:39:38.781068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:38.781068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:38.781068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:38.781068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:38.781068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:38.781068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:38.781068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:38.781068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:38.781068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:38.781068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:38.781068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:38.781068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:38.781068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:38.781068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:38.781068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:38.782117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:38.782117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:38.782117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:38.782117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:38.782117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:38.782117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:38.971053 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:38.971545 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:38.971830 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:38.973390 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:38.973880 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:38.974214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:38.975301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:38.975301 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:38.975301 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:38.975301 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:38.975301 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:38.975301 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:38.975301 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:38.975301 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:38.977159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:38.977159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:38.977159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:38.977159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:38.977159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:38.977159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:39:38.977159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:38.977159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:38.977159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:38.977159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:38.977159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:38.977159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:38.977795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:38.977795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:38.977795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:38.977795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:38.977795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:38.977795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:38.977795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:38.977795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:38.977795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:38.977795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:38.977795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:38.977795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:38.977795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:38.978472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:38.978472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:38.979697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:38.979697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:38.979697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:38.979697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:38.979697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:38.979697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:38.979697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:38.979697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:38.979697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:38.979697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:38.979697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:38.979697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:38.979697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:38.979697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:38.979697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:38.979697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:38.980469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:39:38.980469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:38.980469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:38.980469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:38.980469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:38.980469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:38.980469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:38.980469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:38.980469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:38.980469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:38.980469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:38.980469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:38.980469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:38.980469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:38.980469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:38.980469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:38.980469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:38.980469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:38.980469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:38.982252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:38.982252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:38.982252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:39.005570 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:39.006223 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:39.006342 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:39.006472 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:39:39.007038 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:39.007298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:39.007992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:39.007992 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:39.007992 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:39.007992 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:39.007992 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:39.007992 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:39.007992 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:39.007992 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:39.009022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:39.009022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:39:39.009022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:39.009022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:39.009022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:39.009022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:39.009022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:39.009022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:39.009022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:39.009022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:39.009022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:39.009022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:39:39.009022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:39.009022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:39:39.009022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:39:39.009627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:39.009627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:39:39.009627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:39.009627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:39:39.009627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:39:39.009627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:39.009627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:39:39.009627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:39.009627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:39:39.009947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:39.009947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:39.009947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:39.009947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:39.009947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:39.009947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:39:39.009947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:39:39.009947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:39:39.009947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:39.009947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:39.009947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:39.009947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:39.010462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:39.010462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:39.010462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:39.010462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:39.010462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:39.010462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:39.010462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:39.010462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:39.010462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:39.010462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:39.010462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:39.010462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:39.010462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:39.010462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:39.010889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:39.010889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:39.010889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:39.010889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:39.011145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:39.011145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:39.011145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:39.011145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:39.011145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:39.011145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:39.011145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:39.011145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:39.011145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:39.011145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:39.011145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:39.011145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:39.011145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:39.011145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:39.011145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:39.011145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:39.011642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:39:39.011642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:39:39.011642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:39:39.011642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:39.011642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:39.011642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:39.011642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:39.011642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:39.011642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:39.011642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:39.011642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:39.011642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:39.011642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:39.011642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:39.011642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:39.011642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:39.011642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:39.011642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:39.012193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:39.012193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:39.012193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:39.012193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:39.012193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:39.012193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:39.012193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:39.015006 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:39.015485 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:39.015626 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:39.015822 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:39.016695 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:39.016925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:39.017890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:39.017890 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:39.017890 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:39.017890 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:39.017890 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:39.017890 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:39.017890 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:39.017890 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:39.018977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:39.018977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:39:39.018977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:39.018977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:39.018977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:39.018977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:39.018977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:39.018977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:39.018977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:39.018977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:39.018977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:39.018977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:39:39.018977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:39.018977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:39:39.018977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:39:39.019547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:39.019547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:39:39.019547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:39.019547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:39:39.019547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:39:39.019547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:39.019547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:39:39.019547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:39.019547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:39:39.022645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:39.022645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:39.022645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:39.022645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:39.022645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:39.022645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:39:39.022645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:39:39.022645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:39.022645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:39.022645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:39.022645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:39.023061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:39.023061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:39.023061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:39.023061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:39.023061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:39.023061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:39.023061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:39.023061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:39.023061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:39.023061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:39.023061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:39.023061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:39.023061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:39.023061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:39.023485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:39.023485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:39.023485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:39.023485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:39.023485 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:39.023758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:39.023758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:39.023758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:39.023758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:39.023758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:39.023758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:39.023758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:39.023758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:39.023758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:39.023758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:39.023758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:39.023758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:39.023758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:39.023758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:39.023758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:39.023758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:39.023758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:39:39.024485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:39:39.024485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:39.024485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:39.024485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:39.024485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:39.024485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:39.024485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:39.024485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:39.024485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:39.024485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:39.024485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:39.024485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:39.024485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:39.024485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:39.024485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:39.024485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:39.024485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:39.024485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:39.024485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:39.025052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:39.025052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:39.025052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:39.025052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:39.025052 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:39.025052 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:39.025052 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:39.025052 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:39.025521 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:39.025570 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:39.026337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:39.026337 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:39.026337 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:39.026337 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:39.026337 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:39.026337 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:39.026337 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:39.026337 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:39.027164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:39.027164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:39.027164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:39.027164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:39.027164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:39.027164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:39:39.027164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:39.027164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:39.027164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:39.027164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:39.027164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:39.027164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:39.027553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:39.027553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:39.027553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:39.027553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:39.027553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:39.027553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:39.027553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:39.027553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:39.027553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:39.027553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:39.027553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:39.027553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:39.027553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:39.027987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:39.027987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:39.028059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:39.028059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:39.028059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:39.028059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:39.028059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:39.028059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:39.028059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:39.028059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:39.028059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:39.028059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:39.028059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:39.028059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:39.028059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:39.028059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:39.028059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:39.028059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:39.028564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:39:39.028564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:39.028564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:39.028564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:39.028564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:39.028564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:39.028564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:39.028564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:39.028564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:39.028564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:39.028564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:39.028564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:39.028564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:39.028564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:39.028564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:39.028564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:39.028564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:39.028564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:39.028564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:39.029114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:39.029114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:39.029114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:43.737051 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:43.737870 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:43.738063 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:43.739244 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:43.773525 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:43.773957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:43.774945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:43.774945 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:43.774945 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:43.774945 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:43.774945 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:43.774945 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:43.774945 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:43.774945 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:43.775991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:43.775991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:43.775991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:43.775991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:43.775991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:43.775991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:39:43.775991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:43.775991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:43.775991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:43.775991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:43.775991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:43.775991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:43.776913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:43.776913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:43.776913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:43.776913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:43.776913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:43.776913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:43.776913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:43.776913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:43.776913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:43.776913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:43.776913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:43.776913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:43.776913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:43.777559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:43.777559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:43.777559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:43.777559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:43.777559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:43.777559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:43.777559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:43.777559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:43.777559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:43.777559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:43.777559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:43.777559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:43.777559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:43.777559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:43.777559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:43.778128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:43.778128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:43.778128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:43.778128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:39:43.778128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:43.778128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:43.778128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:43.778128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:43.778128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:43.778128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:43.778128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:43.778128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:43.778128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:43.778128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:43.778128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:43.778128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:43.778128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:43.778128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:43.778128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:43.778800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:43.778800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:43.778800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:43.778800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:43.778800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:43.778800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:43.973065 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:43.973065 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:43.973065 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:43.974260 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:43.974677 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:43.974963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:43.976090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:43.976090 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:43.976090 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:43.976090 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:43.976090 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:43.976090 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:43.976090 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:43.976090 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:43.977838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:43.977838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:43.977838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:43.977838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:43.977838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:43.977838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:39:43.977838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:43.977838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:43.977838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:43.977838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:43.977838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:43.977838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:43.978371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:43.978371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:43.978371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:43.978371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:43.978371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:43.978371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:43.978371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:43.978371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:43.978371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:43.978371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:43.978371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:43.978371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:43.978371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:43.978937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:43.978937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:43.979724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:43.979724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:43.979724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:43.979724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:43.979724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:43.979724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:43.979724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:43.979724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:43.979724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:43.979724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:43.979724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:43.979724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:43.979724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:43.979724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:43.979724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:43.979724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:43.980355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:39:43.980355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:43.980355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:43.980355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:43.980355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:43.980355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:43.980355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:43.980355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:43.980355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:43.980355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:43.980355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:43.980355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:43.980355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:43.980355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:43.980355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:43.980355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:43.980355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:43.980355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:43.980355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:43.980895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:43.980895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:43.980895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:44.006280 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:44.006723 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:44.006832 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:44.007045 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:39:44.008364 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:44.008820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:44.009904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:44.009904 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:44.009904 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:44.009904 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:44.009904 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:44.009904 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:44.009904 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:44.009904 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:44.011135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:44.011135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:39:44.011135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:44.011135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:44.011135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:44.011135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:44.011135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:44.011135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:44.011135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:44.011135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:44.011135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:44.011135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:39:44.011135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:44.011135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:39:44.011135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:39:44.011875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:44.011875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:39:44.011875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:44.011875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:39:44.011875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:39:44.011875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:44.011875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:39:44.011875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:44.011875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:39:44.012406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:44.012406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:44.012406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:44.012406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:44.012406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:44.012406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:39:44.012406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:39:44.012406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:39:44.012406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:44.012406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:44.012406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:44.012406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:44.012792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:44.012792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:44.012792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:44.012792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:44.012792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:44.012792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:44.012792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:44.012792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:44.012792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:44.012792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:44.012792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:44.012792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:44.012792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:44.012792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:44.013144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:44.013144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:44.013144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:44.013144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:44.013409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:44.013409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:44.013409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:44.013409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:44.013409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:44.013409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:44.013409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:44.013409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:44.013409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:44.013409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:44.013409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:44.013409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:44.013409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:44.013409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:44.013409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:44.013409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:44.013847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:39:44.013847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:39:44.013847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:39:44.013847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:44.013847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:44.013847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:44.013847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:44.013847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:44.013847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:44.013847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:44.013847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:44.013847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:44.013847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:44.013847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:44.013847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:44.013847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:44.013847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:44.013847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:44.014293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:44.014293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:44.014293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:44.014293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:44.014293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:44.014293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:44.014293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:44.015592 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:44.015960 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:44.016077 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:44.016203 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:44.016893 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:44.017345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:44.018237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:44.018237 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:44.018237 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:44.018237 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:44.018237 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:44.018237 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:44.018237 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:44.018237 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:44.019022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:44.019022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:39:44.019022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:44.019022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:44.019022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:44.019022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:44.019022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:44.019022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:44.019022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:44.019022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:44.019022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:44.019022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:39:44.019022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:44.019022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:39:44.019022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:39:44.019414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:44.019414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:39:44.019414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:44.019414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:39:44.019414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:39:44.019414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:44.019414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:39:44.019414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:44.019414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:39:44.019934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:44.019934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:44.019934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:44.019934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:44.019934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:44.019934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:39:44.019934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:39:44.019934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:44.019934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:44.019934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:44.019934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:44.020403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:44.020403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:44.020403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:44.020403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:44.020403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:44.020403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:44.020403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:44.020403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:44.020403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:44.020403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:44.020403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:44.020403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:44.020403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:44.020403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:44.020802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:44.020802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:44.020802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:44.020802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:44.021012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:44.021012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:44.021012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:44.021012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:44.021012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:44.021012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:44.021012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:44.021012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:44.021012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:44.021012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:44.021012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:44.021012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:44.021012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:44.021012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:44.021012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:44.021012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:44.021012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:39:44.021560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:39:44.021560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:44.021560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:44.021560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:44.021560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:44.021560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:44.021560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:44.021560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:44.021560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:44.021560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:44.021560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:44.021560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:44.021560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:44.021560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:44.021560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:44.021560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:44.021560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:44.021560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:44.021560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:44.022095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:44.022095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:44.022095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:44.022095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:44.024252 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:44.024546 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:44.024696 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:44.024906 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:44.025454 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:44.025850 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:44.026085 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:44.026896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:44.026896 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:44.026896 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:44.026896 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:44.026896 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:44.026896 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:44.026896 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:44.026896 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:44.027782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:44.027782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:44.027782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:44.027782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:44.027782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:44.027782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:39:44.027782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:44.027782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:44.027782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:44.027782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:44.027782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:44.027782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:44.028119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:44.028119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:44.028119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:44.028119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:44.028119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:44.028119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:44.028119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:44.028119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:44.028119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:44.028119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:44.028119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:44.028119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:44.028119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:44.028509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:44.028509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:44.028812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:44.028812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:44.028812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:44.028812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:44.028812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:44.028812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:44.028812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:44.028812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:44.028812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:44.028812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:44.028812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:44.028812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:44.028812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:44.028812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:44.028812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:44.028812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:44.029262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:39:44.029262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:44.029262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:44.029262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:44.029262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:44.029262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:44.029262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:44.029262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:44.029262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:44.029262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:44.029262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:44.029262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:44.029262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:44.029262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:44.029262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:44.029262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:44.029262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:44.029262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:44.029262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:44.029829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:44.029829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:44.029829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:48.737970 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:48.738899 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:48.738899 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:48.740063 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:48.772918 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:48.773468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:48.774492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:48.774492 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:48.774492 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:48.774492 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:48.774492 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:48.774492 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:48.774492 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:48.774492 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:48.775566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:48.775566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:48.775566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:48.775566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:48.775566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:48.775566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:39:48.775566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:48.775566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:48.775566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:48.775566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:48.775566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:48.775566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:48.776659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:48.776659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:48.776659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:48.776659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:48.776659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:48.776659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:48.776659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:48.776659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:48.776659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:48.776659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:48.776659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:48.776659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:48.776659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:48.777555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:48.777555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:48.777555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:48.777555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:48.777555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:48.777555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:48.777555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:48.777555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:48.777555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:48.777555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:48.777555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:48.777555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:48.777555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:48.777555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:48.777555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:48.778454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:48.778454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:48.778454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:48.778454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:39:48.778454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:48.778454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:48.778454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:48.778454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:48.778454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:48.778454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:48.778454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:48.778454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:48.778454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:48.778454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:48.778454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:48.778454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:48.778454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:48.778454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:48.778454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:48.779522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:48.779522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:48.779522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:48.779522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:48.779522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:48.779522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:48.972843 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:48.973361 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:48.973508 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:48.974782 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:48.975265 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:48.975639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:48.976696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:48.976696 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:48.976696 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:48.976696 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:48.976696 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:48.976696 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:48.976696 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:48.976696 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:48.977860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:48.977860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:48.977860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:48.977860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:48.977860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:48.977860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:39:48.977860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:48.977860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:48.977860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:48.977860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:48.977860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:48.977860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:48.978718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:48.978718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:48.978718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:48.978718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:48.978718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:48.978718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:48.978718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:48.978718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:48.978718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:48.978718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:48.978718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:48.978718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:48.978718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:48.979242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:48.979242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:48.979242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:48.979242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:48.979242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:48.979242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:48.979242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:48.979242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:48.979242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:48.979242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:48.979242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:48.979242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:48.979242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:48.979242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:48.979242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:48.979902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:48.979902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:48.979902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:48.979902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:39:48.979902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:48.979902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:48.979902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:48.979902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:48.979902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:48.979902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:48.979902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:48.979902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:48.979902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:48.979902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:48.979902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:48.979902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:48.979902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:48.979902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:48.979902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:48.980739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:48.980739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:48.980739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:48.980739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:48.980739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:48.980739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:49.006809 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:49.007148 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:49.007286 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:49.007436 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:39:49.008243 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:49.008558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:49.009332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:49.009332 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:49.009332 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:49.009332 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:49.009332 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:49.009332 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:49.009332 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:49.009332 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:49.010180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:49.010180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:39:49.010180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:49.010180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:49.010180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:49.010180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:49.010180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:49.010180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:49.010180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:49.010180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:49.010180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:49.010180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:39:49.010180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:49.010180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:39:49.010180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:39:49.011069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:49.011069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:39:49.011069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:49.011069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:39:49.011069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:39:49.011069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:49.011069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:39:49.011069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:49.011069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:39:49.011069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:49.011069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:49.011069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:49.011069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:49.011069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:49.011869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:39:49.011869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:39:49.011869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:39:49.011869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:49.011869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:49.011869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:49.011869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:49.011869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:49.011869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:49.011869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:49.011869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:49.011869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:49.011869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:49.012481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:49.012481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:49.012481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:49.012481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:49.012481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:49.012481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:49.012481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:49.012481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:49.012481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:49.012481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:49.012481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:49.012481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:49.013024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:49.013024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:49.013024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:49.013024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:49.013024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:49.013024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:49.013024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:49.013024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:49.013024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:49.013024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:49.013024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:49.013024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:49.013024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:49.013024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:49.013024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:49.013024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:49.013751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:39:49.013751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:39:49.013751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:39:49.013751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:49.013751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:49.013751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:49.013751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:49.013751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:49.013751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:49.013751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:49.013751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:49.013751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:49.013751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:49.013751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:49.013751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:49.013751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:49.013751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:49.013751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:49.014564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:49.014564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:49.014564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:49.014564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:49.014564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:49.014564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:49.014564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:49.016097 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:49.016371 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:49.016481 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:49.016602 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:49.017048 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:49.017281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:49.017870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:49.017870 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:49.017870 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:49.017870 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:49.017870 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:49.017870 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:49.017870 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:49.017870 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:49.018629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:49.018629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:39:49.018629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:49.018629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:49.018629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:49.018629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:49.018629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:49.018629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:49.018629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:49.018629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:49.018629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:49.018629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:39:49.018629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:49.018629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:39:49.018629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:39:49.019347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:49.019347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:39:49.019347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:49.019347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:39:49.019347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:39:49.019347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:49.019347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:39:49.019347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:49.019347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:39:49.019630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:49.019630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:49.019630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:49.019630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:49.019630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:49.019630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:39:49.019630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:39:49.019630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:49.019630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:49.019630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:49.019630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:49.020018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:49.020018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:49.020018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:49.020018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:49.020018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:49.020018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:49.020018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:49.020018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:49.020018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:49.020018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:49.020018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:49.020018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:49.020018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:49.020018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:49.020384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:49.020384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:49.020384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:49.020384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:49.020630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:49.020630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:49.020630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:49.020630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:49.020630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:49.020630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:49.020630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:49.020630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:49.020630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:49.020630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:49.020630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:49.020630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:49.020630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:49.020630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:49.020630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:49.020630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:49.020630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:39:49.021125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:39:49.021125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:49.021125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:49.021125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:49.021125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:49.021125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:49.021125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:49.021125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:49.021125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:49.021125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:49.021125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:49.021125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:49.021125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:49.021125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:49.021125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:49.021125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:49.021125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:49.021125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:49.021125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:49.021636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:49.021636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:49.021636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:49.021636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:49.024758 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:49.025057 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:49.025165 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:49.025312 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:49.025885 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:49.026267 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:49.026506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:49.027340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:49.027340 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:49.027340 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:49.027340 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:49.027340 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:49.027340 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:49.027340 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:49.027340 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:49.028196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:49.028196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:49.028196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:49.028196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:49.028196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:49.028196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:39:49.028196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:49.028196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:49.028196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:49.028196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:49.028196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:49.028196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:49.028573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:49.028573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:49.028573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:49.028573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:49.028573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:49.028573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:49.028573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:49.028573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:49.028573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:49.028573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:49.028573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:49.028573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:49.028573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:49.028953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:49.028953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:49.029159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:49.029159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:49.029159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:49.029159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:49.029159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:49.029159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:49.029159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:49.029159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:49.029159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:49.029159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:49.029159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:49.029159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:49.029159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:49.029159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:49.029159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:49.029159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:49.029626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:39:49.029626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:49.029626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:49.029626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:49.029626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:49.029626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:49.029626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:49.029626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:49.029626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:49.029626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:49.029626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:49.029626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:49.029626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:49.029626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:49.029626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:49.029626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:49.029626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:49.029626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:49.029626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:49.030165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:49.030165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:49.030165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:53.738629 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:53.739273 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:53.739273 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:53.740199 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:53.781437 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:53.781982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:53.783438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:53.783438 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:53.783438 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:53.783438 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:53.783438 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:53.783438 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:53.783438 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:53.783438 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:53.784753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.044 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:53.784753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:53.784753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:53.784753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:53.784753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:53.784753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:39:53.784753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:53.784753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:53.784753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:53.784753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:53.784753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:53.784753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:53.785459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:53.785459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:53.785459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:53.785459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:53.785459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:53.785459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:53.785459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:53.785459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:53.785459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:53.785459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:53.785459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:53.785459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:53.785459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:53.786191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:53.786191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:53.786191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:53.786191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:53.786191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:53.786191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:53.786191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:53.786191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:53.786191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:53.786191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:53.786191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:53.786191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:53.786191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:53.786191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:53.786191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:53.787031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:53.787031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:53.787031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:53.787031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:39:53.787031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:53.787031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:53.787031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:53.787031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:53.787031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:53.787031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:53.787031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:53.787031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:53.787031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:53.787031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:53.787031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:53.787031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:53.787031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:53.787031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:53.787031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:53.789043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:53.789043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:53.789043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:53.789043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:53.789043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:53.789043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:53.973540 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:53.974347 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:53.974564 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:53.976145 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:53.976543 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:53.977056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:53.979833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:53.979833 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:53.979833 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:53.979833 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:53.979833 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:53.979833 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:53.979833 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:53.979833 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:53.980660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:53.980660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:53.980660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:53.980660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:53.980660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:53.980660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:39:53.980660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:53.980660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:53.980660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:53.980660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:53.980660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:53.980660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:53.981324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:53.981324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:53.981324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:53.981324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:53.981324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:53.981324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:53.981324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:53.981324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:53.981324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:53.981324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:53.981324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:53.981324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:53.981324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:53.981936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:53.981936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:53.982844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:53.982844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:53.982844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:53.982844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:53.982844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:53.982844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:53.982844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:53.982844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:53.982844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:53.982844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:53.982844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:53.982844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:53.982844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:53.982844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:53.982844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:53.982844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:53.983582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:39:53.983582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:53.983582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:53.983582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:53.983582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:53.983582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:53.983582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:53.983582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:53.983582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:53.983582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:53.983582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:53.983582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:53.983582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:53.983582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:53.983582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:53.983582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:53.983582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:53.983582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:53.983582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:53.984635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:53.984635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:53.984635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:54.007564 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:54.007826 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:54.008196 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:54.008322 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:39:54.009807 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:54.010442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:54.011651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:54.011651 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:54.011651 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:54.011651 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:54.011651 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:54.011651 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:54.011651 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:54.011651 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:54.013248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:54.013248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:39:54.013248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:54.013248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:54.013248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:54.013248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:54.013248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:54.013248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:54.013248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:54.013248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:54.013248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:54.013248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:39:54.013248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:54.013248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:39:54.013248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:39:54.013822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:54.013822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:39:54.013822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:54.013822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:39:54.013822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:39:54.013822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:54.013822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:39:54.013822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:54.013822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:39:54.014792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:54.014792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:54.014792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:54.014792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:54.014792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:54.014792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:39:54.014792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:39:54.014792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:39:54.014792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:54.014792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:54.014792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:54.014792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:54.016282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:54.016282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:54.016282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:54.016282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:54.016282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:54.016282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:54.016282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:54.016282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:54.016282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:54.016282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:54.016282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:54.016282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:54.016282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:54.016282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:54.017095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:54.017095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:54.017095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:54.017095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:54.017095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:54.017095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:54.017095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:54.017095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:54.017095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:54.017095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:54.017095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:54.017095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:54.017095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:54.017095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:54.018532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:54.018532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:54.018532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:54.018532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:54.018532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:54.018532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:54.018532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:39:54.018532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:39:54.018532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:39:54.018532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:54.018532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:54.018532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:54.018532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:54.018532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:54.018532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:54.018532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:54.018532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:54.018532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:54.018532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:54.019741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:54.019741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:54.019741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:54.019741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:54.019741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:54.019741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:54.019741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:54.019741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:54.019741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:54.019741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:54.019741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:54.019741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:54.019741 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:54.019741 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:54.021027 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:54.021027 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:54.021027 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:54.021027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:54.021027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:54.021027 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:54.021027 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:54.021027 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:54.021027 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:54.021027 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:54.021027 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:54.021027 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:54.021503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:54.021503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:39:54.021503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:54.021503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:54.021503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:54.021503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:54.021503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:54.021503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:54.021503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:54.021503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:54.021503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:54.021503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:39:54.021503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:54.021503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:39:54.021503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:39:54.022249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:54.022249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:39:54.022249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:54.022249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:39:54.022249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:39:54.022249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:54.022249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:39:54.022249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:54.022249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:39:54.022249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:54.022249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:54.022249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:54.022249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:54.022249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:54.022839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:39:54.022839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:39:54.022839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:54.022839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:54.022839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:54.022839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:54.022839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:54.022839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:54.022839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:54.022839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:54.022839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:54.022839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:54.022839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:54.023324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:54.023324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:54.023324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:54.023324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:54.023324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:54.023324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:54.023324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:54.023324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:54.023324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:54.023324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:54.023324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:54.023324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:54.023817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:54.023817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:54.023817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:54.023817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:54.023817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:54.023817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:54.023817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:54.023817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:54.023817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:54.023817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:54.023817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:54.023817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:54.023817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:54.023817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:54.023817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:54.023817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:39:54.023817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:39:54.023817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:54.023817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:54.024712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:54.024712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:54.024712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:54.024712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:54.024712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:54.024712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:54.024712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:54.024712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:54.024712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:54.024712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:54.024712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:54.024712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:54.024712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:54.024712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:54.024712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:54.024712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:54.024712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:54.024712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:54.024712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:54.024712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:54.025200 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:54.025441 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:54.025530 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:54.025718 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:54.026254 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:54.027513 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:54.027513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:54.027994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:54.027994 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:54.027994 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:54.027994 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:54.027994 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:54.027994 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:54.027994 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:54.027994 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:54.028830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:54.028830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:54.028830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:54.028830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:54.028830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:54.028830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:39:54.028830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:54.028830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:54.028830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:54.028830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:54.028830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:54.028830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:54.029187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:54.029187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:54.029187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:54.029187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:54.029187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:54.029187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:54.029187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:54.029187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:54.029187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:54.029187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:54.029187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:54.029187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:54.029187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:54.029527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:54.029527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:54.030047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:54.030047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:54.030047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:54.030047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:54.030047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:54.030047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:54.030047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:54.030047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:54.030047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:54.030047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:54.030047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:54.030047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:54.030047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:54.030047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:54.030047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:54.030047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:54.030534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:39:54.030534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:54.030534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:54.030534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:54.030534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:54.030534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:54.030534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:54.030534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:54.030534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:54.030534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:54.030534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:54.030534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:54.030534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:54.030534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:54.030534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:54.030534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:54.030534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:54.030534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:54.030534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:54.031758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:54.031758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:54.031758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:58.762554 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:58.764269 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:58.764269 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:58.765341 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:58.808715 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:58.809068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:58.810024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:58.810024 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:58.810024 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:58.810024 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:58.810024 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:58.810024 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:58.810024 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:58.810024 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:58.811450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:58.811450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:58.811450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:58.811450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:58.811450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:58.811450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:39:58.811450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:58.811450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:58.811450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:58.811450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:58.811450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:58.811450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:58.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:58.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:58.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:58.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:58.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:58.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:58.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:58.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:58.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:58.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:58.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:58.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:58.812126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:58.812808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:58.812808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:58.812808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:58.812808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:58.812808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:58.812808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:58.812808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:58.812808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:58.812808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:58.812808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:58.812808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:58.812808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:58.812808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:58.812808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:58.812808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:58.813561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:58.813561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:58.813561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:58.813561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:39:58.813561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:58.813561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:58.813561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:58.813561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:58.813561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:58.813561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:58.813561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:58.813561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:58.813561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:58.813561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:58.813561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:58.813561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:58.813561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:58.813561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:58.813561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:58.814537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:58.814537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:58.814537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:58.814537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:58.814537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:58.814537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:58.974261 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:58.975014 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:58.975214 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:58.976624 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:58.977080 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:58.977338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:58.978417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:58.978417 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:58.978417 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:58.978417 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:58.978417 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:58.978417 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:58.978417 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:58.978417 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:58.980128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:58.980128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:58.980128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:58.980128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:58.980128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:58.980128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:39:58.980128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:58.980128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:58.980128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:58.980128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:58.980128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:58.980128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:58.980740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:58.980740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:58.980740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:58.980740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:58.980740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:58.980740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:58.980740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:58.980740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:58.980740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:58.980740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:58.980740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:58.980740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:58.980740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:58.981297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:58.981297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:58.981910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:58.981910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:58.981910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:58.981910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:58.981910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:58.981910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:58.981910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:58.981910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:58.981910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:58.981910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:58.981910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:58.981910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:58.981910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:58.981910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:58.981910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:58.981910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:58.982650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:39:58.982650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:58.982650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:58.982650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:58.982650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:58.982650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:58.982650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:58.982650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:58.982650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:58.982650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:58.982650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:58.982650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:58.982650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:58.982650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:58.982650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:58.982650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:58.982650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:58.982650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:58.982650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:58.983402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:58.983402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:58.983402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:59.007826 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:59.008243 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:59.008443 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:59.008646 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:39:59.009489 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:59.009960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:59.011224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:59.011224 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:59.011224 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:59.011224 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:59.011224 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:59.011224 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:59.011224 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:59.011224 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:59.012471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:59.012471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:39:59.012471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:59.012471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:59.012471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:59.012471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:59.012471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:59.012471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:59.012471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:59.012471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:59.012471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:59.012471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:39:59.012471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:59.012471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:39:59.012471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:39:59.013306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:59.013306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:39:59.013306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:59.013306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:39:59.013306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:39:59.013306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:59.013306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:39:59.013306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:59.013306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:39:59.013802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:59.013802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:59.013802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:59.013802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:59.013802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:59.013802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:39:59.013802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:39:59.013802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:39:59.013802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:59.013802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:59.013802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:59.013802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:59.014565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:59.014565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:59.014565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:59.014565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:59.014565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:59.014565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:59.014565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:59.014565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:59.014565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:59.014565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:59.014565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:59.014565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:59.014565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:59.014565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:59.015027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:59.015027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:59.015027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:59.015027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:59.015633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:59.015633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:59.015633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:59.015633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:59.015633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:59.015633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:59.015633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:59.015633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:59.015633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:59.015633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:59.015633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:59.015633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:59.015633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:59.015633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:59.015633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:59.015633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:59.016078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:39:59.016078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:39:59.016078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:39:59.016078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:59.016078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:59.016078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:59.016078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:59.016078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:59.016078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:59.016078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:59.016078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:59.016078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:59.016078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:59.016078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:59.016078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:59.016078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:59.016078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:59.016078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:59.016534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:59.016534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:59.016534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:59.016534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:59.016534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:59.016534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:59.016534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:59.017514 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:59.017933 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:59.018110 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:59.018309 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:59.019020 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:59.019363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:59.020148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:59.020148 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:59.020148 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:59.020148 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:59.020148 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:59.020148 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:59.020148 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:59.020148 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:59.020946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:59.020946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:39:59.020946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:59.020946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:59.020946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:59.020946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:59.020946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:59.020946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:59.020946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:59.020946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:39:59.020946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:59.020946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:39:59.020946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:59.020946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:39:59.020946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:39:59.021360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:59.021360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:39:59.021360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:59.021360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:39:59.021360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:39:59.021360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:59.021360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:39:59.021360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:59.021360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:39:59.021806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:59.021806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:59.021806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:59.021806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:59.021806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:59.021806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:39:59.021806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:39:59.021806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:59.021806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:59.021806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:59.021806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:59.022133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:59.022133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:59.022133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:59.022133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:59.022133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:59.022133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:59.022133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:59.022133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:59.022133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:59.022133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:59.022133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:59.022133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:59.022133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:59.022133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:59.022541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:59.022541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:59.022541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:59.022541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:59.022935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:59.022935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:59.022935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:59.022935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:59.022935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:59.022935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:59.022935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:59.022935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:59.022935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:59.022935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:59.022935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:59.022935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:59.022935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:59.022935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:59.022935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:59.022935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:59.022935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:39:59.023428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:39:59.023428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:39:59.023428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:39:59.023428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:59.023428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:59.023428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:59.023428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:59.023428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:59.023428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:59.023428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:59.023428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:59.023428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:59.023428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:59.023428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:59.023428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:59.023428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:59.023428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:59.023428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:59.023428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:59.023942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:59.023942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:59.023942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:59.023942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:39:59.025535 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:39:59.025850 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:39:59.025979 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:39:59.026149 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:39:59.026659 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:59.027082 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:39:59.027333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:59.028142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:39:59.028142 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:59.028142 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:39:59.028142 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:39:59.028142 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:59.028142 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:39:59.028142 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:59.028142 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:39:59.028990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:59.028990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:39:59.028990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:59.028990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:39:59.028990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:59.028990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:39:59.028990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:59.028990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:59.028990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:59.028990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:59.028990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:59.028990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:59.029329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:39:59.029329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:59.029329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:39:59.029329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:59.029329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:39:59.029329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:39:59.029329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:59.029329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:39:59.029329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:59.029329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:39:59.029329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:39:59.029329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:59.029329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:39:59.029694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:59.029694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:39:59.029962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:59.029962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:39:59.029962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:39:59.029962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:39:59.029962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:39:59.029962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:39:59.029962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:59.029962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:39:59.029962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:39:59.029962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:59.029962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:39:59.029962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:39:59.029962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:39:59.029962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:39:59.029962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:39:59.029962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:39:59.030438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:39:59.030438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:39:59.030438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:59.030438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:59.030438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:39:59.030438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:39:59.030438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:39:59.030438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:39:59.030438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:39:59.030438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:39:59.030438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:39:59.030438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:39:59.030438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:39:59.030438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:39:59.030438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:39:59.030438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:39:59.030438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:39:59.030438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:39:59.030438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:39:59.030964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:39:59.030964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:39:59.030964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:03.763836 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:03.764637 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:03.764637 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:03.764995 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:03.798893 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:03.798893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:03.799653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:03.799653 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:03.799653 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:03.799653 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:03.799653 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:03.799653 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:03.799653 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:03.799653 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:03.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:03.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:03.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:03.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:03.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:03.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:40:03.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:03.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:03.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:03.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:03.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:03.800553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:03.800830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:03.800830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:03.800830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:03.800830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:03.800830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:03.800830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:03.800830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:03.800830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:03.800830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:03.800830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:03.800830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:03.800830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:03.800830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:03.801081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:03.801081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:03.801576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:03.801576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:03.801576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:03.801576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:03.801576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:03.801576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:03.801576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:03.801576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:03.801576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:03.801576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:03.801576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:03.801576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:03.801576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:03.801576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:03.801576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:03.801576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:03.801905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:40:03.801905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:03.801905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:03.801905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:03.801905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:03.801905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:03.801905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:03.801905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:03.801905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:03.801905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:03.801905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:03.801905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:03.801905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:03.801905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:03.801905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:03.801905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:03.801905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:03.801905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:03.801905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:03.802245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:03.802245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:03.802245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:03.974986 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:03.975516 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:03.975711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:03.977287 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:03.977617 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:03.977875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:03.978764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:03.978764 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:03.978764 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:03.978764 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:03.978764 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:03.978764 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:03.978764 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:03.978764 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:03.979981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:03.979981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:03.979981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:03.979981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:03.979981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:03.979981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:40:03.979981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:03.979981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:03.979981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:03.979981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:03.979981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:03.979981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:03.980712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:03.980712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:03.980712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:03.980712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:03.980712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:03.980712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:03.980712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:03.980712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:03.980712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:03.980712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:03.980712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:03.980712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:03.980712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:03.981370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:03.981370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:03.981370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:03.981370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:03.981370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:03.981370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:03.981370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:03.981370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:03.981370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:03.981370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:03.981370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:03.981370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:03.981370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:03.981370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:03.981370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:40:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:03.982444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:03.983526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:03.983526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:03.983526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:03.983526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:03.983526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:03.983526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:04.008239 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:04.008565 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:04.008733 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:04.008896 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:40:04.009810 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:04.010118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:04.011011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:04.011011 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:04.011011 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:04.011011 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:04.011011 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:04.011011 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:04.011011 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:04.011011 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:04.012030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:04.012030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:40:04.012030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:04.012030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:04.012030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:04.012030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:04.012030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:04.012030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:04.012030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:04.012030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:04.012030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:04.012030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:40:04.012030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:04.012030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:40:04.012030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:40:04.012806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:04.012806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:40:04.012806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:04.012806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:40:04.012806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:40:04.012806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:04.012806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:40:04.012806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:04.012806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:40:04.013254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:04.013254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:04.013254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:04.013254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:04.013254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:04.013254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:40:04.013254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:40:04.013254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:40:04.013254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:04.013254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:04.013254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:04.013254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:04.013864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:04.013864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:04.013864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:04.013864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:04.013864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:04.013864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:04.013864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:04.013864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:04.013864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:04.013864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:04.013864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:04.013864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:04.013864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:04.013864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:04.014468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:04.014468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:04.014468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:04.014468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:04.014971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:04.014971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:04.014971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:04.014971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:04.014971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:04.014971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:04.014971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:04.014971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:04.014971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:04.014971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:04.014971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:04.014971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:04.014971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:04.014971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:04.014971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:04.014971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:04.015994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:40:04.015994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:40:04.015994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:40:04.015994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:04.015994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:04.015994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:04.015994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:04.015994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:04.015994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:04.015994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:04.015994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:04.015994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:04.015994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:04.015994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:04.015994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:04.015994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:04.015994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:04.015994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:04.016510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:04.016510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:04.016510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:04.016510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:04.016510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:04.016510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:04.016510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:04.018056 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:04.018360 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:04.018472 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:04.018601 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:04.019292 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:04.019692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:04.020534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:04.020534 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:04.020534 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:04.020534 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:04.020534 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:04.020534 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:04.020534 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:04.020534 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:04.021310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:04.021310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:40:04.021310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:04.021310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:04.021310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:04.021310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:04.021310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:04.021310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:04.021310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:04.021310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:04.021310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:04.021310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:40:04.021310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:04.021310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:40:04.021310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:40:04.021709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:04.021709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:40:04.021709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:04.021709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:40:04.021709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:40:04.021709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:04.021709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:40:04.021709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:04.021709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:40:04.022211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:04.022211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:04.022211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:04.022211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:04.022211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:04.022211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:40:04.022211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:40:04.022211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:04.022211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:04.022211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:04.022211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:04.022534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:04.022534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:04.022534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:04.022534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:04.022534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:04.022534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:04.022534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:04.022534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:04.022534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:04.022534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:04.022534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:04.022534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:04.022534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:04.022534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:04.022944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:04.022944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:04.022944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:04.022944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:04.023656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:04.023656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:04.023656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:04.023656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:04.023656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:04.023656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:04.023656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:04.023656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:04.023656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:04.023656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:04.023656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:04.023656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:04.023656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:04.023656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:04.023656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:04.023656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:04.023656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:40:04.024181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:40:04.024181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:04.024181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:04.024181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:04.024181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:04.024181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:04.024181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:04.024181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:04.024181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:04.024181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:04.024181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:04.024181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:04.024181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:04.024181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:04.024181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:04.024181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:04.024181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:04.024181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:04.024181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:04.024726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:04.024726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:04.024726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:04.024726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:04.025880 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:04.026137 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:04.026251 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:04.026405 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:04.027087 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:04.027384 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:04.027558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:04.028285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:04.028285 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:04.028285 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:04.028285 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:04.028285 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:04.028285 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:04.028285 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:04.028285 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:04.029298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:04.029298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:04.029298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:04.029298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:04.029298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:04.029298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:40:04.029298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:04.029298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:04.029298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:04.029298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:04.029298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:04.029298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:04.029688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:04.029688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:04.029688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:04.029688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:04.029688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:04.029688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:04.029688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:04.029688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:04.029688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:04.029688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:04.029688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:04.029688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:04.029688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:04.030053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:04.030053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:04.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:04.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:04.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:04.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:04.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:04.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:04.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:04.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:04.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:04.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:04.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:04.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:04.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:04.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:04.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:04.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:04.030769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:40:04.030769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:04.030769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:04.030769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:04.030769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:04.030769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:04.030769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:04.030769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:04.030769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:04.030769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:04.030769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:04.030769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:04.030769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:04.030769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:04.030769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:04.030769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:04.030769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:04.030769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:04.030769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:04.031311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:04.031311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:04.031311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:08.764161 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:08.764922 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:08.765063 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:08.766447 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:08.768589 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=226 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:40:08.840758 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:08.840758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:08.840758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:08.840758 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:08.840758 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:08.840758 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:08.840758 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:08.840758 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:08.840758 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:08.840758 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:08.840758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:08.840758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:08.840758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:08.841559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:08.841559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:08.841559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:40:08.841559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:08.841559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:08.841559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:08.841559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:08.841559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:08.841559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:08.841559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:08.841559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:08.841559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:08.841559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:08.841559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:08.842892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:08.842892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:08.842892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:08.842892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:08.842892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:08.842892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:08.842892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:08.842892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:08.842892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:08.842892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:08.842892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:08.842892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:08.843617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:08.843617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:08.843617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:08.843617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:08.843617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:08.843617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:08.843617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:08.843617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:08.843617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:08.843617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:08.843617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:08.843617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:08.843617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:08.843617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:08.843617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:40:08.843617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:08.843617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:08.843617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:08.843617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:08.845213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:08.845213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:08.845213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:08.845213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:08.845213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:08.845213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:08.845213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:08.845213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:08.845213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:08.845213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:08.845213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:08.845213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:08.845213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:08.845213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:08.845213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:08.845213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:08.845213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:08.975730 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:08.976357 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:08.976542 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:08.977437 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:08.977737 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:08.977891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:08.978581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:08.978581 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:08.978581 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:08.978581 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:08.978581 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:08.978581 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:08.978581 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:08.978581 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:08.980184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:08.980184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:08.980184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:08.980184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:08.980184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:08.980184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:40:08.980184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:08.980184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:08.980184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:08.980184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:08.980184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:08.980184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:08.981044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:08.981044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:08.981044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:08.981044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:08.981044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:08.981044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:08.981044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:08.981044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:08.981044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:08.981044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:08.981044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:08.981044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:08.981044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:08.981415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:08.981415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:08.981854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:08.981854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:08.981854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:08.981854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:08.981854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:08.981854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:08.981854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:08.981854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:08.981854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:08.981854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:08.981854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:08.981854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:08.981854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:08.981854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:08.981854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:08.981854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:08.982299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:40:08.982299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:08.982299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:08.982299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:08.982299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:08.982299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:08.982299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:08.982299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:08.982299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:08.982299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:08.982299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:08.982299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:08.982299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:08.982299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:08.982299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:08.982299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:08.982299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:08.982299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:08.982299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:08.982814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:08.982814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:08.982814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:09.008735 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:09.009283 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:09.009465 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:09.009655 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:40:09.011069 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:09.011495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:09.012555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:09.012555 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:09.012555 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:09.012555 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:09.012555 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:09.012555 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:09.012555 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:09.012555 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:09.013906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:09.013906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:40:09.013906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:09.013906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:09.013906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:09.013906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:09.013906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:09.013906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:09.013906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:09.013906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:09.013906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:09.013906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:40:09.013906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:09.013906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:40:09.013906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:40:09.014957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:09.014957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:40:09.014957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:09.014957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:40:09.014957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:40:09.014957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:09.014957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:40:09.014957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:09.014957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:40:09.015358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:09.015358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:09.015358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:09.015358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:09.015358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:09.015358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:40:09.015358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:40:09.015358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:40:09.015358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:09.015358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:09.015358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:09.015358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:09.015941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:09.015941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:09.015941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:09.015941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:09.015941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:09.015941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:09.015941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:09.015941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:09.015941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:09.015941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:09.015941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:09.015941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:09.015941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:09.015941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:09.016479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:09.016479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:09.016479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:09.016479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:09.017286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:09.017286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:09.017286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:09.017286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:09.017286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:09.017286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:09.017286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:09.017286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:09.017286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:09.017286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:09.017286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:09.017286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:09.017286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:09.017286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:09.017286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:09.017286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:09.018227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:40:09.018227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:40:09.018227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:40:09.018227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:09.018227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:09.018227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:09.018227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:09.018227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:09.018227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:09.018227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:09.018227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:09.018227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:09.018227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:09.018227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:09.018227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:09.018227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:09.018227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:09.018227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:09.019247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:09.019247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:09.019247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:09.019247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:09.019247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:09.019247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:09.019247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:09.019247 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:09.019247 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:09.019247 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:09.019946 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:09.019946 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:09.019946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:09.020217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:09.020217 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:09.020217 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:09.020217 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:09.020217 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:09.020217 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:09.020217 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:09.020217 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:09.020805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:09.020805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:40:09.020805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:09.020805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:09.020805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:09.020805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:09.020805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:09.020805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:09.020805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:09.020805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:09.020805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:09.020805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:40:09.020805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:09.020805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:40:09.020805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:40:09.021959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:09.021959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:40:09.021959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:09.021959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:40:09.021959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:40:09.021959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:09.021959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:40:09.021959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:09.021959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:40:09.021959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:09.021959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:09.021959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:09.021959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:09.021959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:09.022739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:40:09.022739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:40:09.022739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:09.022739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:09.022739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:09.022739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:09.022739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:09.022739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:09.022739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:09.022739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:09.022739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:09.022739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:09.022739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:09.023084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:09.023084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:09.023084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:09.023084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:09.023084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:09.023084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:09.023084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:09.023084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:09.023084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:09.023084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:09.023084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:09.023084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:09.023414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:09.023414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:09.023414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:09.023414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:09.023414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:09.023414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:09.023414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:09.023414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:09.023414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:09.023414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:09.023414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:09.023414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:09.023414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:09.023414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:09.023414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:09.023414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:40:09.023414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:40:09.023414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:09.023414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:09.023983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:09.023983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:09.023983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:09.023983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:09.023983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:09.023983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:09.023983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:09.023983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:09.023983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:09.023983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:09.023983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:09.023983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:09.023983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:09.023983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:09.023983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:09.023983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:09.023983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:09.023983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:09.023983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:09.023983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:09.026287 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:09.026560 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:09.026667 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:09.026832 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:09.027358 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:09.027833 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:09.028114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:09.029130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:09.029130 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:09.029130 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:09.029130 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:09.029130 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:09.029130 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:09.029130 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:09.029130 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:09.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:09.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:09.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:09.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:09.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:09.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:40:09.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:09.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:09.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:09.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:09.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:09.030292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:09.030524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:09.030524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:09.030524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:09.030524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:09.030524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:09.030524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:09.030524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:09.030524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:09.030524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:09.030524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:09.030524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:09.030524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:09.030524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:09.030775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:09.030775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:09.031370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:09.031370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:09.031370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:09.031370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:09.031370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:09.031370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:09.031370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:09.031370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:09.031370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:09.031370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:09.031370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:09.031370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:09.031370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:09.031370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:09.031370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:09.031370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:09.031707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:40:09.031707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:09.031707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:09.031707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:09.031707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:09.031707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:09.031707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:09.031707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:09.031707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:09.031707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:09.031707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:09.031707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:09.031707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:09.031707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:09.031707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:09.031707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:09.031707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:09.031707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:09.031707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:09.032054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:09.032054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:09.032054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:13.764965 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:13.765902 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:13.765902 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:13.766468 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:13.800877 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:13.801158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:13.801863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:13.801863 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:13.801863 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:13.801863 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:13.801863 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:13.801863 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:13.801863 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:13.801863 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:13.802895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:13.802895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:13.802895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:13.802895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:13.802895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:13.802895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:40:13.802895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:13.802895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:13.802895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:13.802895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:13.802895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:13.802895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:13.803568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:13.803568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:13.803568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:13.803568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:13.803568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:13.803568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:13.803568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:13.803568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:13.803568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:13.803568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:13.803568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:13.803568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:13.803568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:13.804163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:13.804163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:13.804163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:13.804163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:13.804163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:13.804163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:13.804163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:13.804163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:13.804163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:13.804163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:13.804163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:13.804163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:13.804163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:13.804163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:13.804163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:13.804950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:13.804950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:13.804950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:13.804950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:40:13.804950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:13.804950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:13.804950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:13.804950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:13.804950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:13.804950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:13.804950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:13.804950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:13.804950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:13.804950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:13.804950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:13.804950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:13.804950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:13.804950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:13.804950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:13.805837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:13.805837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:13.805837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:13.805837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:13.805837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:13.805837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:13.976484 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:13.977185 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:13.977414 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:13.978588 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:13.978991 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:13.979225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:13.980235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:13.980235 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:13.980235 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:13.980235 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:13.980235 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:13.980235 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:13.980235 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:13.980235 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:13.981928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:13.981928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:13.981928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:13.981928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:13.981928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:13.981928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:40:13.981928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:13.981928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:13.981928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:13.981928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:13.981928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:13.981928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:13.982503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:13.982503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:13.982503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:13.982503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:13.982503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:13.982503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:13.982503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:13.982503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:13.982503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:13.982503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:13.982503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:13.982503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:13.982503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:13.983107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:13.983107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:13.983796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:13.983796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:13.983796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:13.983796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:13.983796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:13.983796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:13.983796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:13.983796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:13.983796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:13.983796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:13.983796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:13.983796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:13.983796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:13.983796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:13.983796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:13.983796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:13.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:40:13.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:13.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:13.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:13.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:13.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:13.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:13.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:13.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:13.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:13.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:13.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:13.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:13.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:13.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:13.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:13.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:13.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:13.984482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:13.985022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:13.985022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:13.985022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:14.009320 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:14.009690 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:14.009921 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:14.010101 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:40:14.010592 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:14.010918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:14.011691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:14.011691 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:14.011691 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:14.011691 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:14.011691 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:14.011691 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:14.011691 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:14.011691 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:14.012936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:14.012936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:40:14.012936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:14.012936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:14.012936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:14.012936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:14.012936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:14.012936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:14.012936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:14.012936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:14.012936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:14.012936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:40:14.012936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:14.012936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:40:14.012936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:40:14.013343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:14.013343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:40:14.013343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:14.013343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:40:14.013343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:40:14.013343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:14.013343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:40:14.013343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:14.013343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:40:14.014222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:14.014222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:14.014222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:14.014222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:14.014222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:14.014222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:40:14.014222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:40:14.014222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:40:14.014222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:14.014222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:14.014222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:14.014222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:14.014579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:14.014579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:14.014579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:14.014579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:14.014579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:14.014579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:14.014579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:14.014579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:14.014579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:14.014579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:14.014579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:14.014579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:14.014579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:14.014579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:14.014960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:14.014960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:14.014960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:14.014960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:14.015896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:14.015896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:14.015896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:14.015896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:14.015896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:14.015896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:14.015896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:14.015896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:14.015896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:14.015896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:14.015896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:14.015896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:14.015896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:14.015896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:14.015896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:14.015896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:14.016358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:40:14.016358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:40:14.016358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:40:14.016358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:14.016358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:14.016358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:14.016358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:14.016358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:14.016358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:14.016358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:14.016358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:14.016358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:14.016358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:14.016358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:14.016358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:14.016358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:14.016358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:14.016358 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:14.016844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:14.016844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:14.016844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:14.016844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:14.016844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:14.016844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:14.016844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:14.018955 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:14.019303 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:14.019490 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:14.019692 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:14.020238 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:14.020614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:14.021421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:14.021421 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:14.021421 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:14.021421 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:14.021421 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:14.021421 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:14.021421 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:14.021421 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:14.022635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:14.022635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:40:14.022635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:14.022635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:14.022635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:14.022635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:14.022635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:14.022635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:14.022635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:14.022635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:14.022635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:14.022635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:40:14.022635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:14.022635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:40:14.022635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:40:14.023079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:14.023079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:40:14.023079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:14.023079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:40:14.023079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:40:14.023079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:14.023079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:40:14.023079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:14.023079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:40:14.024179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:14.024179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:14.024179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:14.024179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:14.024179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:14.024179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:40:14.024179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:40:14.024179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:14.024179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:14.024179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:14.024179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:14.024507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:14.024507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:14.024507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:14.024507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:14.024507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:14.024507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:14.024507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:14.024507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:14.024507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:14.024507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:14.024507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:14.024507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:14.024507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:14.024507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:14.024894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:14.024894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:14.024894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:14.024894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:14.025253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:14.025253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:14.025253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:14.025253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:14.025253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:14.025253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:14.025253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:14.025253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:14.025253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:14.025253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:14.025253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:14.025253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:14.025253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:14.025253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:14.025253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:14.025253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:14.025253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:40:14.025761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:40:14.025761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:14.025761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:14.025761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:14.025761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:14.025761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:14.025761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:14.025761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:14.025761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:14.025761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:14.025761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:14.025761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:14.025761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:14.025761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:14.025761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:14.025761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:14.025761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:14.025761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:14.025761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:14.026274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:14.026274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:14.026274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:14.026274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:14.026758 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:14.027024 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:14.027130 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:14.027272 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:14.027770 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:14.028171 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:14.028402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:14.029184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:14.029184 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:14.029184 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:14.029184 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:14.029184 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:14.029184 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:14.029184 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:14.029184 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:14.029988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:14.029988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:14.029988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:14.029988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:14.029988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:14.029988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:40:14.029988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:14.029988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:14.029988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:14.029988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:14.029988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:14.029988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:14.030336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:14.030336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:14.030336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:14.030336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:14.030336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:14.030336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:14.030336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:14.030336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:14.030336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:14.030336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:14.030336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:14.030336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:14.030336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:14.030727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:14.030727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:14.030917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:14.030917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:14.030917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:14.030917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:14.030917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:14.030917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:14.030917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:14.030917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:14.030917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:14.030917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:14.030917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:14.030917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:14.030917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:14.030917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:14.030917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:14.030917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:14.031597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:40:14.031597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:14.031597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:14.031597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:14.031597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:14.031597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:14.031597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:14.031597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:14.031597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:14.031597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:14.031597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:14.031597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:14.031597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:14.031597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:14.031597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:14.031597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:14.031597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:14.031597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:14.031597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:14.032156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:14.032156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:14.032156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:18.765601 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:18.766458 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:18.766458 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:18.767961 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:18.799339 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:18.799774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:18.800795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:18.800795 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:18.800795 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:18.800795 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:18.800795 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:18.800795 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:18.800795 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:18.800795 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:18.801528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:18.801528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:18.801528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:18.801528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:18.801528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:18.801528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:40:18.801528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:18.801528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:18.801528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:18.801528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:18.801528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:18.801528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:18.802389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:18.802389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:18.802389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:18.802389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:18.802389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:18.802389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:18.802389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:18.802389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:18.802389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:18.802389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:18.802389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:18.802389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:18.802389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:18.803417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:18.803417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:18.803417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:18.803417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:18.803417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:18.803417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:18.803417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:18.803417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:18.803417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:18.803417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:18.803417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:18.803417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:18.803417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:18.803417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:18.803417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:18.804658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:18.804658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:18.804658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:18.804658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:40:18.804658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:18.804658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:18.804658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:18.804658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:18.804658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:18.804658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:18.804658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:18.804658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:18.804658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:18.804658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:18.804658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:18.804658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:18.804658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:18.804658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:18.804658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:18.806382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:18.806382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:18.806382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:18.806382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:18.806382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:18.806382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:18.977208 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:18.977852 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:18.978043 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:18.979001 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:18.979198 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:18.979309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:18.980066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:18.980066 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:18.980066 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:18.980066 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:18.980066 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:18.980066 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:18.980066 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:18.980066 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:18.981837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:18.981837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:18.981837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:18.981837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:18.981837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:18.981837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:40:18.981837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:18.981837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:18.981837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:18.981837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:18.981837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:18.981837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:18.982415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:18.982415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:18.982415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:18.982415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:18.982415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:18.982415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:18.982415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:18.982415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:18.982415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:18.982415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:18.982415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:18.982415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:18.982415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:18.983055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:18.983055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:18.983055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:18.983055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:18.983055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:18.983055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:18.983055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:18.983055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:18.983055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:18.983055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:18.983055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:18.983055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:18.983055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:18.983055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:18.983055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:18.983877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:18.983877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:18.983877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:18.983877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:40:18.983877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:18.983877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:18.983877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:18.983877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:18.983877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:18.983877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:18.983877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:18.983877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:18.983877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:18.983877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:18.983877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:18.983877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:18.983877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:18.983877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:18.983877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:18.985079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:18.985079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:18.985079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:18.985079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:18.985079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:18.985079 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:19.010117 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:19.010348 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:19.010698 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:19.010905 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:40:19.011413 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:19.011758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:19.012611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:19.012611 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:19.012611 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:19.012611 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:19.012611 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:19.012611 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:19.012611 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:19.012611 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:19.014245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:19.014245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:40:19.014245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:19.014245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:19.014245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:19.014245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:19.014245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:19.014245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:19.014245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:19.014245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:19.014245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:19.014245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:40:19.014245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:19.014245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:40:19.014245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:40:19.015561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:19.015561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:40:19.015561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:19.015561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:40:19.015561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:40:19.015561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:19.015561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:40:19.015561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:19.015561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:40:19.015561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:19.015561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:19.015561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:19.015561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:19.015561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:19.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:40:19.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:40:19.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:40:19.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:19.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:19.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:19.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:19.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:19.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:19.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:19.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:19.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:19.016556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:19.017466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:19.017466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:19.017466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:19.017466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:19.017466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:19.017466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:19.017466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:19.017466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:19.017466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:19.017466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:19.017466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:19.017466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:19.018132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:19.018132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:19.018132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:19.018132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:19.018132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:19.018132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:19.018132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:19.018132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:19.018132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:19.018132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:19.018132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:19.018132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:19.018132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:19.018132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:19.018132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:19.018132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:19.019263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:40:19.019263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:40:19.019263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:40:19.019263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:19.019263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:19.019263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:19.019263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:19.019263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:19.019263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:19.019263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:19.019263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:19.019263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:19.019263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:19.019263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:19.019263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:19.019263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:19.019263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:19.019263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:19.020182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:19.020182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:19.020182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:19.020182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:19.020182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:19.020182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:19.020182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:19.020182 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:19.020182 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:19.020182 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:19.021477 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:19.021477 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:19.021477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:19.021477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:19.021477 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:19.021477 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:19.021477 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:19.021477 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:19.021477 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:19.021477 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:19.021477 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:19.022164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:19.022164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:40:19.022164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:19.022164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:19.022164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:19.022164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:19.022164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:19.022164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:19.022164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:19.022164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:19.022164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:19.022164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:40:19.022164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:19.022164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:40:19.022164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:40:19.022483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:19.022483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:40:19.022483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:19.022483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:40:19.022483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:40:19.022483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:19.022483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:40:19.022483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:19.022483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:40:19.023018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:19.023018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:19.023018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:19.023018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:19.023018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:19.023018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:40:19.023018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:40:19.023018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:19.023018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:19.023018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:19.023018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:19.023269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:19.023269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:19.023269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:19.023269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:19.023269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:19.023269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:19.023269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:19.023269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:19.023269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:19.023269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:19.023269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:19.023269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:19.023269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:19.023269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:19.023549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:19.023549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:19.023549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:19.023549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:19.024230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:19.024230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:19.024230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:19.024230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:19.024230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:19.024230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:19.024230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:19.024230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:19.024230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:19.024230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:19.024230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:19.024230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:19.024230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:19.024230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:19.024230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:19.024230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:19.024230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:40:19.024599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:40:19.024599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:19.024599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:19.024599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:19.024599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:19.024599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:19.024599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:19.024599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:19.024599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:19.024599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:19.024599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:19.024599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:19.024599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:19.024599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:19.024599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:19.024599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:19.024599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:19.024599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:19.024599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:19.025087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:19.025087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:19.025087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:19.025087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:19.027180 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:19.027461 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:19.027568 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:19.027739 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:19.028152 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:19.028482 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:19.028730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:19.029414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:19.029414 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:19.029414 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:19.029414 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:19.029414 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:19.029414 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:19.029414 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:19.029414 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:19.030230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:19.030230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:19.030230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:19.030230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:19.030230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:19.030230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:40:19.030230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:19.030230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:19.030230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:19.030230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:19.030230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:19.030230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:19.030623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:19.030623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:19.030623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:19.030623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:19.030623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:19.030623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:19.030623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:19.030623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:19.030623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:19.030623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:19.030623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:19.030623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:19.030623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:19.031038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:19.031038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:19.031171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:19.031171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:19.031171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:19.031171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:19.031171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:19.031171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:19.031171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:19.031171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:19.031171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:19.031171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:19.031171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:19.031171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:19.031171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:19.031171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:19.031171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:19.031171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:19.031663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:40:19.031663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:19.031663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:19.031663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:19.031663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:19.031663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:19.031663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:19.031663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:19.031663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:19.031663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:19.031663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:19.031663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:19.031663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:19.031663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:19.031663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:19.031663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:19.031663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:19.031663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:19.031663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:19.032220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:19.032220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:19.032220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:23.766624 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:23.767538 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:23.767744 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:23.769448 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:23.803937 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:23.804404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:23.805664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:23.805664 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:23.805664 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:23.805664 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:23.805664 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:23.805664 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:23.805664 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:23.805664 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:23.806698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:23.806698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:23.806698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:23.806698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:23.806698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:23.806698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:40:23.806698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:23.806698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:23.806698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:23.806698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:23.806698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:23.806698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:23.807328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:23.807328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:23.807328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:23.807328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:23.807328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:23.807328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:23.807328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:23.807328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:23.807328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:23.807328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:23.807328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:23.807328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:23.807328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:23.807993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:23.807993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:23.807993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:23.807993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:23.807993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:23.807993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:23.807993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:23.807993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:23.807993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:23.807993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:23.807993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:23.807993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:23.807993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:23.807993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:23.807993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:23.808774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:23.808774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:23.808774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:23.808774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:40:23.808774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:23.808774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:23.808774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:23.808774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:23.808774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:23.808774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:23.808774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:23.808774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:23.808774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:23.808774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:23.808774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:23.808774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:23.808774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:23.808774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:23.808774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:23.809836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:23.809836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:23.809836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:23.809836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:23.809836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:23.809836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:23.977927 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:23.978954 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:23.979175 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:23.980637 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:23.981089 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:23.981341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:23.982342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:23.982342 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:23.982342 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:23.982342 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:23.982342 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:23.982342 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:23.982342 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:23.982342 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:23.984563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:23.984563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:23.984563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:23.984563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:23.984563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:23.984563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:40:23.984563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:23.984563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:23.984563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:23.984563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:23.984563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:23.984563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:23.985189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:23.985189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:23.985189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:23.985189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:23.985189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:23.985189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:23.985189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:23.985189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:23.985189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:23.985189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:23.985189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:23.985189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:23.985189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:23.985765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:23.985765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:23.987200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:23.987200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:23.987200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:23.987200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:23.987200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:23.987200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:23.987200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:23.987200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:23.987200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:23.987200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:23.987200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:23.987200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:23.987200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:23.987200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:23.987200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:23.987200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:23.988034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:40:23.988034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:23.988034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:23.988034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:23.988034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:23.988034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:23.988034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:23.988034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:23.988034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:23.988034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:23.988034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:23.988034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:23.988034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:23.988034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:23.988034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:23.988034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:23.988034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:23.988034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:23.988034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:23.988923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:23.988923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:23.988923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:24.010422 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:24.011483 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:24.011483 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:24.011483 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:40:24.012103 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:24.012365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:24.013041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:24.013041 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:24.013041 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:24.013041 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:24.013041 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:24.013041 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:24.013041 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:24.013041 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:24.013923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:24.013923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:40:24.013923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:24.013923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:24.013923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:24.013923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:24.013923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:24.013923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:24.013923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:24.013923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:24.013923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:24.013923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:40:24.013923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:24.013923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:40:24.013923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:40:24.014614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:24.014614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:40:24.014614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:24.014614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:40:24.014614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:40:24.014614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:24.014614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:40:24.014614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:24.014614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:40:24.015063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:24.015063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:24.015063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:24.015063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:24.015063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:24.015063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:40:24.015063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:40:24.015063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:40:24.015063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:24.015063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:24.015063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:24.015063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:24.015641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:24.015641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:24.015641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:24.015641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:24.015641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:24.015641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:24.015641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:24.015641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:24.015641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:24.015641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:24.015641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:24.015641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:24.015641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:24.015641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:24.016043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:24.016043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:24.016043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:24.016043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:24.016161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:24.016161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:24.016161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:24.016161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:24.016161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:24.016161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:24.016161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:24.016161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:24.016161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:24.016161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:24.016161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:24.016161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:24.016161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:24.016161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:24.016161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:24.016161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:24.016564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:40:24.016564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:40:24.016564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:40:24.016564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:24.016564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:24.016564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:24.016564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:24.016564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:24.016564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:24.016564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:24.016564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:24.016564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:24.016564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:24.016564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:24.016564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:24.016564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:24.016564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:24.016564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:24.017107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:24.017107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:24.017107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:24.017107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:24.017107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:24.017107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:24.017107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:24.019830 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:24.020328 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:24.020519 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:24.020786 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:24.021172 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:24.021328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:24.021849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:24.021849 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:24.021849 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:24.021849 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:24.021849 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:24.021849 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:24.021849 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:24.021849 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:24.023185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:24.023185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:40:24.023185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:24.023185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:24.023185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:24.023185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:24.023185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:24.023185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:24.023185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:24.023185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:24.023185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:24.023185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:40:24.023185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:24.023185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:40:24.023185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:40:24.023641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:24.023641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:40:24.023641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:24.023641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:40:24.023641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:40:24.023641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:24.023641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:40:24.023641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:24.023641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:40:24.024664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:24.024664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:24.024664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:24.024664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:24.024664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:24.024664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:40:24.024664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:40:24.024664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:24.024664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:24.024664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:24.024664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:24.025015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:24.025015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:24.025015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:24.025015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:24.025015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:24.025015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:24.025015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:24.025015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:24.025015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:24.025015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:24.025015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:24.025015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:24.025015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:24.025015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:24.025433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:24.025433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:24.025433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:24.025433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:24.026435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:24.026435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:24.026435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:24.026435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:24.026435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:24.026435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:24.026435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:24.026435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:24.026435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:24.026435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:24.026435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:24.026435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:24.026435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:24.026435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:24.026435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:24.026435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:24.026435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:40:24.026955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:40:24.026955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:24.026955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:24.026955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:24.026955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:24.026955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:24.026955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:24.026955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:24.026955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:24.026955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:24.026955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:24.026955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:24.026955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:24.026955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:24.026955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:24.026955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:24.026955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:24.026955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:24.026955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:24.027503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:24.027503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:24.027503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:24.027503 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:24.027610 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:24.027878 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:24.028047 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:24.028272 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:24.028537 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:24.028716 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:24.028823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:24.029222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:24.029222 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:24.029222 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:24.029222 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:24.029222 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:24.029222 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:24.029222 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:24.029222 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:24.030115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.001 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:24.030115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:24.030115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:24.030115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:24.030115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:24.030115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:40:24.030115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:24.030115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:24.030115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:24.030115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:24.030115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:24.030115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:24.030462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:24.030462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:24.030462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:24.030462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:24.030462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:24.030462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:24.030462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:24.030462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:24.030462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:24.030462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:24.030462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:24.030462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:24.030462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:24.030851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:24.030851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:24.031142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:24.031142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:24.031142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:24.031142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:24.031142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:24.031142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:24.031142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:24.031142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:24.031142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:24.031142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:24.031142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:24.031142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:24.031142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:24.031142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:24.031142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:24.031142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:24.031631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:40:24.031631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:24.031631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:24.031631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:24.031631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:24.031631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:24.031631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:24.031631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:24.031631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:24.031631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:24.031631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:24.031631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:24.031631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:24.031631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:24.031631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:24.031631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:24.031631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:24.031631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:24.031631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:24.032174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:24.032174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:24.032174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:28.767451 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:28.768455 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:28.768551 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:28.769968 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:28.808409 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:28.808646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:28.809455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:28.809455 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:28.809455 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:28.809455 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:28.809455 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:28.809455 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:28.809455 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:28.809455 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:28.810494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:28.810494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:28.810494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:28.810494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:28.810494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:28.810494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:40:28.810494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:28.810494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:28.810494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:28.810494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:28.810494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:28.810494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:28.811246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:28.811246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:28.811246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:28.811246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:28.811246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:28.811246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:28.811246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:28.811246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:28.811246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:28.811246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:28.811246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:28.811246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:28.811246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:28.811834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:28.811834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:28.811834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:28.811834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:28.811834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:28.811834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:28.811834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:28.811834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:28.811834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:28.811834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:28.811834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:28.811834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:28.811834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:28.811834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:28.811834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:28.812478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:28.812478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:28.812478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:28.812478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:40:28.812478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:28.812478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:28.812478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:28.812478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:28.812478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:28.812478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:28.812478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:28.812478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:28.812478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:28.812478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:28.812478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:28.812478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:28.812478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:28.812478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:28.812478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:28.813295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:28.813295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:28.813295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:28.813295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:28.813295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:28.813295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:28.978868 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:28.986649 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:28.986649 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:28.988306 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:28.988644 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:28.988968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:28.991697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:28.991697 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:28.991697 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:28.991697 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:28.991697 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:28.991697 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:28.991697 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:28.991697 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:28.992725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.012 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:28.992725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:28.992725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:28.992725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:28.992725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:28.992725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:40:28.992725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:28.992725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:28.992725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:28.992725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:28.992725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:28.992725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:28.993076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:28.993076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:28.993076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:28.993076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:28.993076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:28.993076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:28.993076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:28.993076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:28.993076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:28.993076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:28.993076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:28.993076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:28.993076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:28.993438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:28.993438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:28.994507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:28.994507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:28.994507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:28.994507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:28.994507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:28.994507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:28.994507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:28.994507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:28.994507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:28.994507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:28.994507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:28.994507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:28.994507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:28.994507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:28.994507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:28.994507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:28.994996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:40:28.994996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:28.994996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:28.994996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:28.994996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:28.994996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:28.994996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:28.994996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:28.994996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:28.994996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:28.994996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:28.994996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:28.994996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:28.994996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:28.994996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:28.994996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:28.994996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:28.994996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:28.994996 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:28.995514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:28.995514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:28.995514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:29.011141 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:29.011616 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:29.011829 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:29.012018 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:40:29.012613 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:29.013030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:29.013778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:29.013778 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:29.013778 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:29.013778 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:29.013778 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:29.013778 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:29.013778 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:29.013778 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:29.015356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:29.015356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:40:29.015356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:29.015356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:29.015356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:29.015356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:29.015356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:29.015356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:29.015356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:29.015356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:29.015356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:29.015356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:40:29.015356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:29.015356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:40:29.015356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:40:29.015984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:29.015984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:40:29.015984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:29.015984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:40:29.015984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:40:29.015984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:29.015984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:40:29.015984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:29.015984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:40:29.016667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:29.016667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:29.016667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:29.016667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:29.016667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:29.016667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:40:29.016667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:40:29.016667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:40:29.016667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:29.016667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:29.016667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:29.016667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:29.017221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:29.017221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:29.017221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:29.017221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:29.017221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:29.017221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:29.017221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:29.017221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:29.017221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:29.017221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:29.017221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:29.017221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:29.017221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:29.017221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:29.017865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:29.017865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:29.017865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:29.017865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:29.018346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:29.018346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:29.018346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:29.018346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:29.018346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:29.018346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:29.018346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:29.018346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:29.018346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:29.018346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:29.018346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:29.018346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:29.018346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:29.018346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:29.018346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:29.018346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:29.026468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:40:29.026468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:40:29.026468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:40:29.026468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:29.026468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:29.026468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:29.026468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:29.026468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:29.026468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:29.026468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:29.026468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:29.026468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:29.026468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:29.026468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:29.026468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:29.026468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:29.026468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:29.026468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:29.026994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:29.026994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:29.026994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:29.026994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:29.026994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:29.026994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:29.026994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:29.026994 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:29.026994 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:29.026994 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:29.027196 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:29.027196 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:29.027196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:29.027196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:29.027196 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:29.027196 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:29.027196 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:29.027196 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:29.027196 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:29.027196 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:29.027196 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:29.027196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:29.027424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:40:29.027424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:29.027424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:29.027424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:29.027424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:29.027424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:29.027424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:29.027424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:29.027424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:29.027424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:29.027424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:40:29.027424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:29.027424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:40:29.027424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:40:29.027424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:29.027424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:40:29.027424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:29.027754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:40:29.027754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:40:29.027754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:29.027754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:40:29.027754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:29.027754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:40:29.027754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:29.027754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:29.027754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:29.027754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:29.027754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:29.027754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:40:29.027754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:40:29.028003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:29.028003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:29.028003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:29.028003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:29.028003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:29.028003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:29.028003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:29.028003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:29.028003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:29.028003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:29.028003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:29.028003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:29.028003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:29.028237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:29.028237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:29.028237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:29.028237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:29.028237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:29.028237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:29.028237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:29.028237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:29.028237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:29.028237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:29.028237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:29.028237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:29.028451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:29.028451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:29.028451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:29.028451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:29.028451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:29.028451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:29.028451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:29.028451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:29.028451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:29.028451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:29.028451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:29.028451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:29.028451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:29.028451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:40:29.028451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:40:29.028451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:29.028451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:29.028451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:29.028451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:29.028810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:29.028810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:29.028810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:29.028810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:29.028810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:29.028810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:29.028810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:29.028810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:29.028810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:29.028810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:29.028810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:29.028810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:29.028810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:29.028810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:29.028810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:29.028810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:29.028810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:29.028810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:29.029209 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:29.029209 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:29.029209 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:29.029209 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:29.029893 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:29.030251 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:29.030485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:29.031178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:29.031178 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:29.031178 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:29.031178 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:29.031178 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:29.031178 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:29.031178 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:29.031178 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:29.031979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:29.031979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:29.031979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:29.031979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:29.031979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:29.031979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:40:29.031979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:29.031979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:29.031979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:29.031979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:29.031979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:29.031979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:29.032228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:29.032228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:29.032228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:29.032228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:29.032228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:29.032228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:29.032228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:29.032228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:29.032228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:29.032228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:29.032228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:29.032228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:29.032228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:29.032489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:29.032489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:29.032946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:29.032946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:29.032946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:29.032946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:29.032946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:29.032946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:29.032946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:29.032946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:29.032946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:29.032946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:29.032946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:29.032946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:29.032946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:29.032946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:29.032946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:29.032946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:29.033249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:40:29.033249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:29.033249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:29.033249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:29.033249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:29.033249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:29.033249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:29.033249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:29.033249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:29.033249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:29.033249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:29.033249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:29.033249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:29.033249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:29.033249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:29.033249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:29.033249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:29.033249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:29.033249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:29.033619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:29.033619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:29.033619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:33.768232 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:33.768984 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:33.769283 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:33.770307 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:33.804614 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:33.805053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:33.806187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:33.806187 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:33.806187 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:33.806187 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:33.806187 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:33.806187 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:33.806187 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:33.806187 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:33.807229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:33.807229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:33.807229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:33.807229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:33.807229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:33.807229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:40:33.807229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:33.807229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:33.807229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:33.807229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:33.807229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:33.807229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:33.807901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:33.807901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:33.807901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:33.807901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:33.807901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:33.807901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:33.807901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:33.807901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:33.807901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:33.807901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:33.807901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:33.807901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:33.807901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:33.808507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:33.808507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:33.808507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:33.808507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:33.808507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:33.808507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:33.808507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:33.808507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:33.808507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:33.808507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:33.808507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:33.808507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:33.808507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:33.808507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:33.808507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:33.809460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:33.809460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:33.809460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:33.809460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:40:33.809460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:33.809460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:33.809460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:33.809460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:33.809460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:33.809460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:33.809460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:33.809460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:33.809460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:33.809460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:33.809460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:33.809460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:33.809460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:33.809460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:33.809460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:33.810514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:33.810514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:33.810514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:33.810514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:33.810514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:33.810514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:33.979532 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:33.980455 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:33.980639 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:33.982063 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:33.982481 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:33.982774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:33.983775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:33.983775 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:33.983775 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:33.983775 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:33.983775 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:33.983775 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:33.983775 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:33.983775 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:33.985578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:33.985578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:33.985578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:33.985578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:33.985578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:33.985578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:40:33.985578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:33.985578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:33.985578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:33.985578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:33.985578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:33.985578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:33.986834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:33.986834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:33.986834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:33.986834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:33.986834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:33.986834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:33.986834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:33.986834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:33.986834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:33.986834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:33.986834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:33.986834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:33.986834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:33.988046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:33.988046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:33.988046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:33.988046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:33.988046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:33.988046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:33.988046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:33.988046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:33.988046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:33.988046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:33.988046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:33.988046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:33.988046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:33.988046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:33.988046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:33.989448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:33.989448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:33.989448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:33.989448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:40:33.989448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:33.989448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:33.989448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:33.989448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:33.989448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:33.989448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:33.989448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:33.989448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:33.989448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:33.989448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:33.989448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:33.989448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:33.989448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:33.989448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:33.989448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:33.990738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:33.990738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:33.990738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:33.990738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:33.990738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:33.990738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:34.011627 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:34.011999 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:34.012129 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:34.012250 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:40:34.020879 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:34.027157 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:34.027522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:34.028527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:34.028527 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:34.028527 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:34.028527 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:34.028527 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:34.028527 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:34.028527 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:34.028527 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:34.029281 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:34.030478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:34.030478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:40:34.030478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:34.030478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:34.030478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:34.030478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:34.030478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:34.030478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:34.030478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:34.030478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:34.030478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:34.030478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:40:34.030478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:34.030478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:40:34.030478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:40:34.031751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:34.031751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:40:34.031751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:34.031751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:40:34.031751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:40:34.031751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:34.031751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:40:34.031751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:34.031751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:40:34.032189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.018 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:34.032189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:34.032189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:34.032189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:34.032189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:34.032189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:40:34.032189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:40:34.032189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:40:34.032189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:34.032189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:34.032189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:34.032189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:34.032787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:34.032787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:34.032787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:34.032787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:34.032787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:34.032787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:34.032787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:34.032787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:34.032787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:34.032787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:34.032787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:34.032787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:34.032787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:34.032787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:34.033365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:34.033365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:34.033365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:34.033365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:34.034454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:34.034454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:34.034454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:34.034454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:34.034454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:34.034454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:34.034454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:34.034454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:34.034454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:34.034454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:34.034454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:34.034454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:34.034454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:34.034454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:34.034454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:34.034454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:34.035797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:40:34.035797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:40:34.035797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:40:34.035797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:34.035797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:34.035797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:34.035797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:34.035797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:34.035797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:34.035797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:34.035797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:34.035797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:34.035797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:34.035797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:34.035797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:34.035797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:34.035797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:34.035797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:34.036584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:34.036584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:34.036584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:34.036584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:34.036584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:34.036584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:34.036584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:34.036584 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:34.036584 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:34.036584 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:34.037070 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:34.037070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:34.037070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:34.037070 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:34.037070 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:34.037070 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:34.037070 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:34.037070 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:34.037070 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:34.037070 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:34.037506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:34.037506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:40:34.037506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:34.037506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:34.037506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:34.037506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:34.037506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:34.037506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:34.037506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:34.037506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:34.037506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:34.037506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:40:34.037506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:34.037506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:40:34.037506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:40:34.038855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:34.038855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:40:34.038855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:34.038855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:40:34.038855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:40:34.038855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:34.038855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:40:34.038855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:34.038855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:40:34.038855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:34.038855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:34.038855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:34.038855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:34.038855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:34.039407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:40:34.039407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:40:34.039407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:34.039407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:34.039407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:34.039407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:34.039407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:34.039407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:34.039407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:34.039407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:34.039407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:34.039407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:34.039407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:34.039935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:34.039935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:34.039935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:34.039935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:34.039935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:34.039935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:34.039935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:34.039935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:34.039935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:34.039935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:34.039935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:34.039935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:34.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:34.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:34.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:34.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:34.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:34.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:34.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:34.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:34.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:34.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:34.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:34.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:34.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:34.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:34.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:34.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:40:34.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:40:34.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:34.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:34.041146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:34.041146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:34.041146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:34.041146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:34.041146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:34.041146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:34.041146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:34.041146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:34.041146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:34.041146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:34.041146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:34.041146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:34.041146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:34.041146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:34.041146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:34.041146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:34.041146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:34.041146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:34.041146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:34.041146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:34.041936 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:34.041936 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:34.041936 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:34.041936 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:34.041936 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:34.041936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:34.042213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:34.042213 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:34.042213 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:34.042213 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:34.042213 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:34.042213 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:34.042213 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:34.042213 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:34.042549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:34.042549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:34.042549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:34.042549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:34.042549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:34.042549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:40:34.042549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:34.042549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:34.042549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:34.042549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:34.042549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:34.042549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:34.042791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:34.042791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:34.042791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:34.042791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:34.042791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:34.042791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:34.042791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:34.042791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:34.042791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:34.042791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:34.042791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:34.042791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:34.042791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:34.043017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:34.043017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:34.043568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:34.043568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:34.043568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:34.043568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:34.043568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:34.043568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:34.043568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:34.043568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:34.043568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:34.043568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:34.043568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:34.043568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:34.043568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:34.043568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:34.043568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:34.043568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:34.043895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:40:34.043895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:34.043895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:34.043895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:34.043895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:34.043895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:34.043895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:34.043895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:34.043895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:34.043895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:34.043895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:34.043895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:34.043895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:34.043895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:34.043895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:34.043895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:34.043895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:34.043895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:34.043895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:34.044243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:34.044243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:34.044243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:38.768969 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:38.769763 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:38.769973 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:38.771377 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:38.813856 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:38.814390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:38.815606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:38.815606 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:38.815606 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:38.815606 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:38.815606 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:38.815606 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:38.815606 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:38.815606 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:38.816602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:38.816602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:38.816602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:38.816602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:38.816602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:38.816602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:40:38.816602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:38.816602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:38.816602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:38.816602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:38.816602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:38.816602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:38.817058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:38.817058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:38.817058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:38.817058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:38.817058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:38.817058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:38.817058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:38.817058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:38.817058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:38.817058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:38.817058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:38.817058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:38.817058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:38.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:38.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:38.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:38.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:38.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:38.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:38.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:38.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:38.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:38.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:38.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:38.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:38.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:38.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:38.817510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:38.818043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:38.818043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:38.818043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:38.818043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:40:38.818043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:38.818043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:38.818043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:38.818043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:38.818043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:38.818043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:38.818043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:38.818043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:38.818043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:38.818043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:38.818043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:38.818043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:38.818043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:38.818043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:38.818043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:38.818692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:38.818692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:38.818692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:38.818692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:38.818692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:38.818692 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:38.980259 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:38.983379 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:38.983565 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:38.985341 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:38.985797 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:38.986142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:38.987333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:38.987333 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:38.987333 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:38.987333 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:38.987333 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:38.987333 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:38.987333 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:38.987333 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:38.989088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:38.989088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:38.989088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:38.989088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:38.989088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:38.989088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:40:38.989088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:38.989088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:38.989088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:38.989088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:38.989088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:38.989088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:38.989505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:38.989505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:38.989505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:38.989505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:38.989505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:38.989505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:38.989505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:38.989505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:38.989505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:38.989505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:38.989505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:38.989505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:38.989505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:38.989903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:38.989903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:38.990378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:38.990378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:38.990378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:38.990378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:38.990378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:38.990378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:38.990378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:38.990378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:38.990378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:38.990378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:38.990378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:38.990378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:38.990378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:38.990378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:38.990378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:38.990378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:38.990977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:40:38.990977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:38.990977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:38.990977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:38.990977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:38.990977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:38.990977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:38.990977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:38.990977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:38.990977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:38.990977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:38.990977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:38.990977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:38.990977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:38.990977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:38.990977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:38.990977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:38.990977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:38.990977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:38.991529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:38.991529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:38.991529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:39.012256 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:39.012909 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:39.013259 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:39.013259 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:40:39.014120 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:39.014862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:39.015392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:39.015392 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:39.015392 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:39.015392 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:39.015392 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:39.015392 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:39.015392 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:39.015392 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:39.016793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:39.016793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:40:39.016793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:39.016793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:39.016793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:39.016793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:39.016793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:39.016793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:39.016793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:39.016793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:39.016793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:39.016793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:40:39.016793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:39.016793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:40:39.016793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:40:39.017267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:39.017267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:40:39.017267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:39.017267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:40:39.017267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:40:39.017267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:39.017267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:40:39.017267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:39.017267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:40:39.019346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:39.019346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:39.019346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:39.019346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:39.019346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:39.019346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:40:39.019346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:40:39.019346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:40:39.019346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:39.019346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:39.019346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:39.019346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:39.019761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:39.019761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:39.019761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:39.019761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:39.019761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:39.019761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:39.019761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:39.019761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:39.019761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:39.019761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:39.019761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:39.019761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:39.019761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:39.019761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:39.020134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:39.020134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:39.020134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:39.020134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:39.020134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:39.020134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:39.020134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:39.020134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:39.020134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:39.020134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:39.020134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:39.020134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:39.020134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:39.020134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:39.020514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:39.020514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:39.020514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:39.020514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:39.020514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:39.020514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:39.020514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:40:39.020514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:40:39.020514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:40:39.020514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:39.020514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:39.020514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:39.020514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:39.020514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:39.020514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:39.020514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:39.020514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:39.020514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:39.020514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:39.021087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:39.021087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:39.021087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:39.021087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:39.021087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:39.021087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:39.021087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:39.021087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:39.021087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:39.021087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:39.021087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:39.021087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:39.034577 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:39.035015 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:39.035015 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:39.035193 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:39.039751 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:39.057506 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:39.058115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:39.059335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:39.059335 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:39.059335 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:39.059335 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:39.059335 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:39.059335 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:39.059335 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:39.059335 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:39.062743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:39.062743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:40:39.062743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:39.062743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:39.062743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:39.062743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:39.062743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:39.062743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:39.062743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:39.062743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:39.062743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:39.062743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:40:39.062743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:39.062743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:40:39.062743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:40:39.063463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:39.063463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:40:39.063463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:39.063463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:40:39.063463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:40:39.063463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:39.063463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:40:39.063463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:39.063463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:40:39.063463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.025 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:39.063463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:39.063463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:39.063463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:39.063463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:39.064089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:40:39.064089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:40:39.064089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:39.064089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:39.064089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:39.064089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:39.064089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:39.064089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:39.064089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:39.064089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:39.064089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:39.064089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:39.064089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:39.064668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:39.064668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:39.064668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:39.064668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:39.064668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:39.064668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:39.064668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:39.064668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:39.064668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:39.064668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:39.064668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:39.064668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:39.065205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:39.065205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:39.065205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:39.065205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:39.065205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:39.065205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:39.065205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:39.065205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:39.065205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:39.065205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:39.065205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:39.065205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:39.065205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:39.065205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:39.065205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:39.065205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:40:39.065205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:40:39.065205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:39.065205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:39.065583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:39.065583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:39.065583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:39.065583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:39.065583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:39.065583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:39.065583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:39.065583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:39.065583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:39.065583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:39.065583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:39.065583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:39.065583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:39.065583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:39.065583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:39.065583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:39.065583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:39.065583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:39.065583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:39.065583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:39.066535 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:39.066535 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:39.066535 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:39.066535 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:39.066535 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:39.066535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:39.066710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:39.066710 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:39.066710 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:39.066710 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:39.066710 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:39.066710 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:39.066710 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:39.066710 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:39.067321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:39.067321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:39.067321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:39.067321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:39.067321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:39.067321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:40:39.067321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:39.067321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:39.067321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:39.067321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:39.067321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:39.067321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:39.067564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:39.067564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:39.067564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:39.067564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:39.067564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:39.067564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:39.067564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:39.067564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:39.067564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:39.067564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:39.067564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:39.067564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:39.067564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:39.067823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:39.067823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:39.069008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:39.069008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:39.069008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:39.069008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:39.069008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:39.069008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:39.069008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:39.069008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:39.069008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:39.069008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:39.069008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:39.069008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:39.069008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:39.069008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:39.069008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:39.069008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:39.069347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:40:39.069347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:39.069347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:39.069347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:39.069347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:39.069347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:39.069347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:39.069347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:39.069347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:39.069347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:39.069347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:39.069347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:39.069347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:39.069347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:39.069347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:39.069347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:39.069347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:39.069347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:39.069347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:39.069698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:39.069698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:39.069698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:43.769572 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:43.770711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:43.770975 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:43.772785 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:43.811228 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:43.811699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:43.812640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:43.812640 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:43.812640 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:43.812640 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:43.812640 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:43.812640 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:43.812640 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:43.812640 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:43.813616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:43.813616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:43.813616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:43.813616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:43.813616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:43.813616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:40:43.813616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:43.813616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:43.813616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:43.813616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:43.813616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:43.813616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:43.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:43.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:43.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:43.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:43.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:43.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:43.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:43.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:43.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:43.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:43.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:43.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:43.814345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:43.814995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:43.814995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:43.814995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:43.814995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:43.814995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:43.814995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:43.814995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:43.814995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:43.814995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:43.814995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:43.814995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:43.814995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:43.814995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:43.814995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:43.814995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:43.815970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:43.815970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:43.815970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:43.815970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:40:43.815970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:43.815970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:43.815970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:43.815970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:43.815970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:43.815970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:43.815970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:43.815970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:43.815970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:43.815970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:43.815970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:43.815970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:43.815970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:43.815970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:43.815970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:43.816750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:43.816750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:43.816750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:43.816750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:43.816750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:43.816750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:43.983393 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:43.984321 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:43.984531 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:43.988023 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:43.988380 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:43.988625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:43.989758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:43.989758 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:43.989758 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:43.989758 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:43.989758 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:43.989758 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:43.989758 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:43.989758 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:43.990873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:43.990873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:43.990873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:43.990873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:43.990873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:43.990873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:40:43.990873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:43.990873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:43.990873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:43.990873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:43.990873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:43.990873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:43.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:43.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:43.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:43.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:43.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:43.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:43.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:43.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:43.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:43.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:43.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:43.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:43.991471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:43.992208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:43.992208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:43.992208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:43.992208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:43.992208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:43.992208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:43.992208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:43.992208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:43.992208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:43.992208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:43.992208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:43.992208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:43.992208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:43.992208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:43.992208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:43.993155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:43.993155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:43.993155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:43.993155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:40:43.993155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:43.993155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:43.993155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:43.993155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:43.993155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:43.993155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:43.993155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:43.993155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:43.993155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:43.993155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:43.993155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:43.993155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:43.993155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:43.993155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:43.993155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:43.994097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:43.994097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:43.994097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:43.994097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:43.994097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:43.994097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:44.012858 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:44.013331 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:44.013506 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:44.013701 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:40:44.014268 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:44.014600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:44.015460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:44.015460 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:44.015460 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:44.015460 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:44.015460 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:44.015460 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:44.015460 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:44.015460 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:44.016260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:44.016260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:40:44.016260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:44.016260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:44.016260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:44.016260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:44.016260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:44.016260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:44.016260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:44.016260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:44.016260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:44.016260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:40:44.016260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:44.016260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:40:44.016260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:40:44.017054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:44.017054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:40:44.017054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:44.017054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:40:44.017054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:40:44.017054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:44.017054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:40:44.017054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:44.017054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:40:44.017567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:44.017567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:44.017567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:44.017567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:44.017567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:44.017567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:40:44.017567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:40:44.017567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:40:44.017567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:44.017567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:44.017567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:44.017567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:44.018260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:44.018260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:44.018260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:44.018260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:44.018260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:44.018260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:44.018260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:44.018260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:44.018260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:44.018260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:44.018260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:44.018260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:44.018260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:44.018260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:44.018941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:44.018941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:44.018941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:44.018941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:44.018941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:44.018941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:44.018941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:44.018941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:44.018941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:44.018941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:44.018941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:44.018941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:44.018941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:44.018941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:44.019659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:44.019659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:44.019659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:44.019659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:44.019659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:44.019659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:44.019659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:40:44.019659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:40:44.019659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:40:44.019659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:44.019659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:44.019659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:44.019659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:44.019659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:44.019659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:44.019659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:44.019659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:44.019659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:44.019659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:44.020704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:44.020704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:44.020704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:44.020704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:44.020704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:44.020704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:44.020704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:44.020704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:44.020704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:44.020704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:44.020704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:44.020704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:44.035032 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:44.035545 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:44.035851 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:44.036094 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:44.037360 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:44.037880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:44.038975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:44.038975 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:44.038975 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:44.038975 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:44.038975 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:44.038975 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:44.038975 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:44.038975 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:44.040564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:44.040564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:40:44.040564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:44.040564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:44.040564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:44.040564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:44.040564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:44.040564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:44.040564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:44.040564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:44.040564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:44.040564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:40:44.040564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:44.040564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:40:44.040564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:40:44.041349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:44.041349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:40:44.041349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:44.041349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:40:44.041349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:40:44.041349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:44.041349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:40:44.041349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:44.041349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:40:44.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:44.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:44.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:44.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:44.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:44.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:40:44.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:40:44.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:44.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:44.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:44.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:44.042818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:44.042818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:44.042818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:44.042818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:44.042818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:44.042818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:44.042818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:44.042818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:44.042818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:44.042818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:44.042818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:44.042818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:44.042818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:44.042818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:44.043528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:44.043528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:44.043528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:44.043528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:44.044180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:44.044180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:44.044180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:44.044180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:44.044180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:44.044180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:44.044180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:44.044180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:44.044180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:44.044180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:44.044180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:44.044180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:44.044180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:44.044180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:44.044180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:44.044180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:44.044180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:40:44.045089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:40:44.045089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:44.045089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:44.045089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:44.045089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:44.045089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:44.045089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:44.045089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:44.045089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:44.045089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:44.045089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:44.045089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:44.045089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:44.045089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:44.045089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:44.045089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:44.045089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:44.045089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:44.045089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:44.046058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:44.046058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:44.046058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:44.046058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:44.105592 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:44.107426 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:44.107426 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:44.107753 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:44.108311 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:44.108588 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:44.108752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:44.109529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:44.109529 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:44.109529 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:44.109529 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:44.109529 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:44.109529 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:44.109529 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:44.109529 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:44.111042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:44.111042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:44.111042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:44.111042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:44.111042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:44.111042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:40:44.111042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:44.111042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:44.111042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:44.111042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:44.111042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:44.111042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:44.111643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:44.111643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:44.111643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:44.111643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:44.111643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:44.111643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:44.111643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:44.111643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:44.111643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:44.111643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:44.111643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:44.111643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:44.111643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:44.112302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:44.112302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:44.112607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:44.112607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:44.112607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:44.112607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:44.112607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:44.112607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:44.112607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:44.112607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:44.112607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:44.112607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:44.112607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:44.112607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:44.112607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:44.112607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:44.112607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:44.112607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:44.113347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:40:44.113347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:44.113347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:44.113347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:44.113347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:44.113347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:44.113347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:44.113347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:44.113347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:44.113347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:44.113347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:44.113347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:44.113347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:44.113347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:44.113347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:44.113347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:44.113347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:44.113347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:44.113347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:44.114178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:44.114178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:44.114178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:48.770491 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:48.771443 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:48.771663 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:48.773064 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:48.805599 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:48.806088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:48.807188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:48.807188 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:48.807188 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:48.807188 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:48.807188 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:48.807188 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:48.807188 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:48.807188 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:48.808474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:48.808474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:48.808474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:48.808474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:48.808474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:48.808474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:40:48.808474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:48.808474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:48.808474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:48.808474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:48.808474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:48.808474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:48.809150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:48.809150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:48.809150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:48.809150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:48.809150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:48.809150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:48.809150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:48.809150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:48.809150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:48.809150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:48.809150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:48.809150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:48.809150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:48.809795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:48.809795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:48.810025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:48.810025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:48.810025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:48.810025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:48.810025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:48.810025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:48.810025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:48.810025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:48.810025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:48.810025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:48.810025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:48.810025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:48.810025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:48.810025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:48.810025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:48.810025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:48.810579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:40:48.810579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:48.810579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:48.810579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:48.810579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:48.810579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:48.810579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:48.810579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:48.810579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:48.810579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:48.810579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:48.810579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:48.810579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:48.810579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:48.810579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:48.810579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:48.810579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:48.810579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:48.810579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:48.811190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:48.811190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:48.811190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:48.984073 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:48.984649 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:48.984887 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:48.986306 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:48.986764 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:48.987044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:48.988133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:48.988133 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:48.988133 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:48.988133 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:48.988133 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:48.988133 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:48.988133 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:48.988133 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:48.989741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:48.989741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:48.989741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:48.989741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:48.989741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:48.989741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:40:48.989741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:48.989741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:48.989741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:48.989741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:48.989741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:48.989741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:48.990445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:48.990445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:48.990445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:48.990445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:48.990445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:48.990445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:48.990445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:48.990445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:48.990445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:48.990445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:48.990445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:48.990445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:48.990445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:48.991095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:48.991095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:48.991430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:48.991430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:48.991430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:48.991430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:48.991430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:48.991430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:48.991430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:48.991430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:48.991430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:48.991430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:48.991430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:48.991430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:48.991430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:48.991430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:48.991430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:48.991430 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:48.992306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:40:48.992306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:48.992306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:48.992306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:48.992306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:48.992306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:48.992306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:48.992306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:48.992306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:48.992306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:48.992306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:48.992306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:48.992306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:48.992306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:48.992306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:48.992306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:48.992306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:48.992306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:48.992306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:48.993248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:48.993248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:48.993248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:49.013341 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:49.013761 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:49.013938 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:49.014121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:40:49.014960 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:49.015332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:49.016404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:49.016404 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:49.016404 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:49.016404 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:49.016404 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:49.016404 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:49.016404 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:49.016404 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:49.017612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:49.017612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:40:49.017612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:49.017612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:49.017612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:49.017612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:49.017612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:49.017612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:49.017612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:49.017612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:49.017612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:49.017612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:40:49.017612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:49.017612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:40:49.017612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:40:49.018581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:49.018581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:40:49.018581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:49.018581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:40:49.018581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:40:49.018581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:49.018581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:40:49.018581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:49.018581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:40:49.019103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:49.019103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:49.019103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:49.019103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:49.019103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:49.019103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:40:49.019103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:40:49.019103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:40:49.019103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:49.019103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:49.019103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:49.019103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:49.019670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:49.019670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:49.019670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:49.019670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:49.019670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:49.019670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:49.019670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:49.019670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:49.019670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:49.019670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:49.019670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:49.019670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:49.019670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:49.019670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:49.020410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:49.020410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:49.020410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:49.020410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:49.020741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:49.020741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:49.020741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:49.020741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:49.020741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:49.020741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:49.020741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:49.020741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:49.020741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:49.020741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:49.020741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:49.020741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:49.020741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:49.020741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:49.020741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:49.020741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:49.021241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:40:49.021241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:40:49.021241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:40:49.021241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:49.021241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:49.021241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:49.021241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:49.021241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:49.021241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:49.021241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:49.021241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:49.021241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:49.021241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:49.021241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:49.021241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:49.021241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:49.021241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:49.021241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:49.021783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:49.021783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:49.021783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:49.021783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:49.021783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:49.021783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:49.021783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:49.035611 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:49.035919 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:49.036032 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:49.036171 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:49.037130 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:49.037513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:49.038323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:49.038323 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:49.038323 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:49.038323 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:49.038323 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:49.038323 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:49.038323 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:49.038323 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:49.039549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:49.039549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:40:49.039549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:49.039549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:49.039549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:49.039549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:49.039549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:49.039549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:49.039549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:49.039549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:49.039549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:49.039549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:40:49.039549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:49.039549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:40:49.039549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:40:49.040343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:49.040343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:40:49.040343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:49.040343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:40:49.040343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:40:49.040343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:49.040343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:40:49.040343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:49.040343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:40:49.040988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:49.040988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:49.040988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:49.040988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:49.040988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:49.040988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:40:49.040988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:40:49.040988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:49.040988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:49.040988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:49.040988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:49.041769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:49.041769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:49.041769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:49.041769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:49.041769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:49.041769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:49.041769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:49.041769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:49.041769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:49.041769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:49.041769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:49.041769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:49.041769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:49.041769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:49.042806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:49.042806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:49.042806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:49.042806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:49.042806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:49.042806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:49.042806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:49.042806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:49.042806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:49.042806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:49.042806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:49.042806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:49.042806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:49.042806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:49.043450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:49.043450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:49.043450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:49.043450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:49.043450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:49.043450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:49.043450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:40:49.043450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:40:49.043450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:49.043450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:49.043450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:49.043450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:49.043450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:49.043450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:49.043450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:49.043450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:49.043450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:49.043450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:49.043450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:49.044311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:49.044311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:49.044311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:49.044311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:49.044311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:49.044311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:49.044311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:49.044311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:49.044311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:49.044311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:49.044311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:49.106662 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:49.107164 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:49.107335 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:49.107611 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:49.109189 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:49.109577 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:49.109734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:49.110569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:49.110569 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:49.110569 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:49.110569 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:49.110569 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:49.110569 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:49.110569 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:49.110569 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:49.112189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:49.112189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:49.112189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:49.112189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:49.112189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:49.112189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:40:49.112189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:49.112189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:49.112189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:49.112189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:49.112189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:49.112189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:49.113015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:49.113015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:49.113015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:49.113015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:49.113015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:49.113015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:49.113015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:49.113015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:49.113015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:49.113015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:49.113015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:49.113015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:49.113015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:49.113805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:49.113805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:49.114344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:49.114344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:49.114344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:49.114344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:49.114344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:49.114344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:49.114344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:49.114344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:49.114344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:49.114344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:49.114344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:49.114344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:49.114344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:49.114344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:49.114344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:49.114344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:49.115123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:40:49.115123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:49.115123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:49.115123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:49.115123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:49.115123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:49.115123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:49.115123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:49.115123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:49.115123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:49.115123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:49.115123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:49.115123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:49.115123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:49.115123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:49.115123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:49.115123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:49.115123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:49.115123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:49.115915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:49.115915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:49.115915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:53.771178 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:53.771893 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:53.772135 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:53.773457 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:53.804514 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:53.805508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:53.805631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:53.805631 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:53.805631 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:53.805631 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:53.805631 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:53.805631 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:53.805631 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:53.805631 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:53.806214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:53.806214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:53.806214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:53.806214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:53.806214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:53.806214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:40:53.806214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:53.806214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:53.806214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:53.806214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:53.806214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:53.806214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:53.806723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:53.806723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:53.806723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:53.806723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:53.806723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:53.806723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:53.806723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:53.806723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:53.806723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:53.806723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:53.806723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:53.806723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:53.806723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:53.807144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:53.807144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:53.807144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:53.807144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:53.807144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:53.807144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:53.807144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:53.807144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:53.807144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:53.807144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:53.807144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:53.807144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:53.807144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:53.807144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:53.807144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:53.807853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:53.807853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:53.807853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:53.807853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:40:53.807853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:53.807853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:53.807853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:53.807853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:53.807853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:53.807853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:53.807853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:53.807853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:53.807853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:53.807853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:53.807853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:53.807853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:53.807853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:53.807853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:53.807853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:53.808580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:53.808580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:53.808580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:53.808580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:53.808580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:53.808580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:53.984908 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:53.985407 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:53.985563 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:53.987086 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:53.987086 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:53.987086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:53.987479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:53.987479 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:53.987479 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:53.987479 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:53.987479 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:53.987479 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:53.987479 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:53.987479 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:53.988609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:53.988609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:53.988609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:53.988609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:53.988609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:53.988609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:40:53.988609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:53.988609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:53.988609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:53.988609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:53.988609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:53.988609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:53.989628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:53.989628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:53.989628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:53.989628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:53.989628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:53.989628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:53.989628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:53.989628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:53.989628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:53.989628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:53.989628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:53.989628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:53.989628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:53.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:53.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:53.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:53.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:53.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:53.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:53.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:53.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:53.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:53.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:53.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:53.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:53.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:53.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:53.990580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:53.991969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:53.991969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:53.991969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:53.991969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:40:53.991969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:53.991969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:53.991969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:53.991969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:53.991969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:53.991969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:53.991969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:53.991969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:53.991969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:53.991969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:53.991969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:53.991969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:53.991969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:53.991969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:53.991969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:53.993711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:53.993711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:53.993711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:53.993711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:53.993711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:53.993711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:54.014115 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:54.014631 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:54.014908 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:54.015191 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:40:54.015789 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:54.016290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:54.017225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:54.017225 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:54.017225 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:54.017225 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:54.017225 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:54.017225 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:54.017225 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:54.017225 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:54.018563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:54.018563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:40:54.018563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:54.018563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:54.018563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:54.018563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:54.018563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:54.018563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:54.018563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:54.018563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:54.018563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:54.018563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:40:54.018563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:54.018563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:40:54.018563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:40:54.019032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:54.019032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:40:54.019032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:54.019032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:40:54.019032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:40:54.019032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:54.019032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:40:54.019032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:54.019032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:40:54.020237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:54.020237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:54.020237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:54.020237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:54.020237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:54.020237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:40:54.020237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:40:54.020237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:40:54.020237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:54.020237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:54.020237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:54.020237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:54.020583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:54.020583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:54.020583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:54.020583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:54.020583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:54.020583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:54.020583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:54.020583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:54.020583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:54.020583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:54.020583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:54.020583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:54.020583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:54.020583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:54.020980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:54.020980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:54.020980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:54.020980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:54.021670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:54.021670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:54.021670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:54.021670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:54.021670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:54.021670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:54.021670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:54.021670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:54.021670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:54.021670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:54.021670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:54.021670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:54.021670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:54.021670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:54.021670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:54.021670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:54.022134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:40:54.022134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:40:54.022134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:40:54.022134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:54.022134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:54.022134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:54.022134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:54.022134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:54.022134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:54.022134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:54.022134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:54.022134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:54.022134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:54.022134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:54.022134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:54.022134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:54.022134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:54.022134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:54.022781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:54.022781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:54.022781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:54.022781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:54.022781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:54.022781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:54.022781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:54.036444 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:54.036444 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:54.036631 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:54.036926 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:54.037412 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:54.037798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:54.038779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:54.038779 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:54.038779 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:54.038779 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:54.038779 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:54.038779 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:54.038779 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:54.038779 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:54.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:54.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:40:54.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:54.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:54.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:54.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:54.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:54.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:54.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:54.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:54.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:54.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:40:54.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:54.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:40:54.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:40:54.040590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:54.040590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:40:54.040590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:54.040590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:40:54.040590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:40:54.040590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:54.040590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:40:54.040590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:54.040590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:40:54.041472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:54.041472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:54.041472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:54.041472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:54.041472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:54.041472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:40:54.041472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:40:54.041472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:54.041472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:54.041472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:54.041472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:54.041967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:54.041967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:54.041967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:54.041967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:54.041967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:54.041967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:54.041967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:54.041967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:54.041967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:54.041967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:54.041967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:54.041967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:54.041967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:54.041967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:54.042464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:54.042464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:54.042464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:54.042464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:54.043378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:54.043378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:54.043378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:54.043378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:54.043378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:54.043378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:54.043378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:54.043378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:54.043378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:54.043378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:54.043378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:54.043378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:54.043378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:54.043378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:54.043378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:54.043378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:54.043378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:40:54.044039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:40:54.044039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:54.044039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:54.044039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:54.044039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:54.044039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:54.044039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:54.044039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:54.044039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:54.044039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:54.044039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:54.044039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:54.044039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:54.044039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:54.044039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:54.044039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:54.044039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:54.044039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:54.044039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:54.044754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:54.044754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:54.044754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:54.044754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:54.107277 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:54.107708 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:54.107888 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:54.108133 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:54.108624 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:54.109115 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:54.109446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:54.110549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:54.110549 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:54.110549 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:54.110549 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:54.110549 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:54.110549 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:54.110549 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:54.110549 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:54.111587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:54.111587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:54.111587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:54.111587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:54.111587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:54.111587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:40:54.111587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:54.111587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:54.111587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:54.111587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:54.111587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:54.111587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:54.112186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:54.112186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:54.112186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:54.112186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:54.112186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:54.112186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:54.112186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:54.112186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:54.112186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:54.112186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:54.112186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:54.112186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:54.112186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:54.113162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:54.113162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:54.113162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:54.113162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:54.113162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:54.113162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:54.113162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:54.113162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:54.113162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:54.113162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:54.113162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:54.113162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:54.113162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:54.113162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:54.113162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:54.113983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:54.113983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:54.113983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:54.113983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:40:54.113983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:54.113983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:54.113983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:54.113983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:54.113983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:54.113983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:54.113983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:54.113983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:54.113983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:54.113983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:54.113983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:54.113983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:54.113983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:54.113983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:54.113983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:54.114962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:54.114962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:54.114962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:54.114962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:54.114962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:54.114962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:58.772711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:58.772711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:58.772711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:58.776697 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:58.811449 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:58.811772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:58.812787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:58.812787 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:58.812787 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:58.812787 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:58.812787 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:58.812787 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:58.812787 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:58.812787 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:58.814465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:58.814465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:58.814465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:58.814465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:58.814465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:58.814465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:40:58.814465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:58.814465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:58.814465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:58.814465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:58.814465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:58.814465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:58.815146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:58.815146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:58.815146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:58.815146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:58.815146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:58.815146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:58.815146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:58.815146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:58.815146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:58.815146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:58.815146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:58.815146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:58.815146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:58.818239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:58.818239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:58.818239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:58.818239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:58.818239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:58.818239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:58.818239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:58.818239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:58.818239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:58.818239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:58.818239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:58.818239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:58.818239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:58.818239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:58.818239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:58.819483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:58.819483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:58.819483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:58.819483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:40:58.819483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:58.819483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:58.819483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:58.819483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:58.819483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:58.819483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:58.819483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:58.819483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:58.819483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:58.819483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:58.819483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:58.819483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:58.819483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:58.819483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:58.819483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:58.820219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:58.820219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:58.820219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:58.820219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:58.820219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:58.820219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:58.985663 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:58.986302 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:58.986400 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:58.987600 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:58.987965 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:58.988146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:58.988897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:58.988897 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:58.988897 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:58.988897 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:58.988897 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:58.988897 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:58.988897 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:58.988897 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:58.990183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:58.990183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:58.990183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:58.990183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:58.990183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:58.990183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:40:58.990183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:58.990183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:58.990183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:58.990183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:58.990183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:58.990183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:58.990788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:58.990788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:58.990788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:58.990788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:58.990788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:58.990788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:58.990788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:58.990788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:58.990788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:58.990788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:58.990788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:58.990788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:58.990788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:58.991311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:58.991311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:58.991497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:58.991497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:58.991497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:58.991497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:58.991497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:58.991497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:58.991497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:58.991497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:58.991497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:58.991497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:58.991497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:58.991497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:58.991497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:58.991497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:58.991497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:58.991497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:58.993071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:40:58.993071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:58.993071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:58.993071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:58.993071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:58.993071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:58.993071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:58.993071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:58.993071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:58.993071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:58.993071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:58.993071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:58.993071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:58.993071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:58.993071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:58.993071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:58.993071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:58.993071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:58.993071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:58.994311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:58.994311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:58.994311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:59.014627 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:59.015097 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:59.015266 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:59.015470 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:40:59.016197 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:59.016531 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:59.017513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:59.017513 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:59.017513 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:59.017513 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:59.017513 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:59.017513 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:59.017513 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:59.017513 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:59.018929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:59.018929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:40:59.018929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:59.018929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:59.018929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:59.018929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:59.018929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:59.018929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:59.018929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:59.018929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:59.018929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:59.018929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:40:59.018929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:59.018929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:40:59.018929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:40:59.020102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:59.020102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:40:59.020102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:59.020102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:40:59.020102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:40:59.020102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:59.020102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:40:59.020102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:59.020102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:40:59.020555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:59.020555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:59.020555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:59.020555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:59.020555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:59.020555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:40:59.020555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:40:59.020555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:40:59.020555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:59.020555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:59.020555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:59.020555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:59.021169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:59.021169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:59.021169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:59.021169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:59.021169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:59.021169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:59.021169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:59.021169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:59.021169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:59.021169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:59.021169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:59.021169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:59.021169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:59.021169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:59.021809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:59.021809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:59.021809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:59.021809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:59.021993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:59.021993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:59.021993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:59.021993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:59.021993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:59.021993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:59.021993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:59.021993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:59.021993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:59.021993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:59.021993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:59.021993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:59.021993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:59.021993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:59.021993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:59.021993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:59.023906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:40:59.023906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:40:59.023906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:40:59.023906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:59.023906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:59.023906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:59.023906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:59.023906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:59.023906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:59.023906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:59.023906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:59.023906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:59.023906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:59.023906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:59.023906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:59.023906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:59.023906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:59.023906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:59.025615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:59.025615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:59.025615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:59.025615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:59.025615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:59.025615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:59.025615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:59.036536 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:59.037057 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:59.037297 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:59.037533 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:59.039403 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:59.039897 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:59.040665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:59.040665 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:59.040665 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:59.040665 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:59.040665 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:59.040665 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:59.040665 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:59.040665 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:59.042272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:59.042272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:40:59.042272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:59.042272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:59.042272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:59.042272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:59.042272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:59.042272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:59.042272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:59.042272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:40:59.042272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:59.042272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:40:59.042272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:59.042272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:40:59.042272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:40:59.043014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:59.043014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:40:59.043014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:59.043014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:40:59.043014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:40:59.043014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:59.043014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:40:59.043014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:59.043014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:40:59.043908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:59.043908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:59.043908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:59.043908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:59.043908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:59.043908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:40:59.043908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:40:59.043908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:59.043908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:59.043908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:59.043908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:59.044478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:59.044478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:59.044478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:59.044478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:59.044478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:59.044478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:59.044478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:59.044478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:59.044478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:59.044478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:59.044478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:59.044478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:59.044478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:59.044478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:59.045094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:59.045094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:59.045094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:59.045094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:59.045860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:59.045860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:59.045860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:59.045860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:59.045860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:59.045860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:59.045860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:59.045860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:59.045860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:59.045860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:59.045860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:59.045860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:59.045860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:59.045860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:59.045860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:59.045860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:59.045860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:40:59.046377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:40:59.046377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:40:59.046377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:40:59.046377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:59.046377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:59.046377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:59.046377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:59.046377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:59.046377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:59.046377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:59.046377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:59.046377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:59.046377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:59.046377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:59.046377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:59.046377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:59.046377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:59.046377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:59.046377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:59.046898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:59.046898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:59.046898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:59.046898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:40:59.107833 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:40:59.108555 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:40:59.108771 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:40:59.109080 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:40:59.110413 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:59.111079 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:40:59.111362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:59.112506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:40:59.112506 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:59.112506 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:40:59.112506 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:40:59.112506 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:59.112506 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:40:59.112506 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:59.112506 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:40:59.114098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:59.114098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:40:59.114098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:59.114098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:40:59.114098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:59.114098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:40:59.114098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:59.114098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:59.114098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:59.114098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:59.114098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:59.114098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:59.114741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:40:59.114741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:59.114741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:40:59.114741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:59.114741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:40:59.114741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:40:59.114741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:59.114741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:40:59.114741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:59.114741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:40:59.114741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:40:59.114741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:59.114741 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:40:59.115265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:59.115265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:40:59.115755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:59.115755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:40:59.115755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:40:59.115755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:40:59.115755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:40:59.115755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:40:59.115755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:59.115755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:40:59.115755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:40:59.115755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:59.115755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:40:59.115755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:40:59.115755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:40:59.115755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:40:59.115755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:40:59.115755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:40:59.116516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:40:59.116516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:40:59.116516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:59.116516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:59.116516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:40:59.116516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:40:59.116516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:40:59.116516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:40:59.116516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:40:59.116516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:40:59.116516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:40:59.116516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:40:59.116516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:40:59.116516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:40:59.116516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:40:59.116516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:40:59.116516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:40:59.116516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:40:59.116516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:40:59.117474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:40:59.117474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:40:59.117474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:03.772487 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:03.773522 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:03.773522 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:03.773935 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:03.813425 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:03.813993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:03.815202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:03.815202 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:03.815202 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:03.815202 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:03.815202 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:03.815202 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:03.815202 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:03.815202 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:03.816139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:03.816139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:03.816139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:03.816139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:03.816139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:03.816139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:41:03.816139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:03.816139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:03.816139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:03.816139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:03.816139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:03.816139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:03.817283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:03.817283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:03.817283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:03.817283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:03.817283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:03.817283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:03.817283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:03.817283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:03.817283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:03.817283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:03.817283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:03.817283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:03.817283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:03.817940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:03.817940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:03.817940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:03.817940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:03.817940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:03.817940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:03.817940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:03.817940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:03.817940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:03.817940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:03.817940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:03.817940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:03.817940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:03.817940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:03.817940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:03.819118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:03.819118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:03.819118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:03.819118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:41:03.819118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:03.819118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:03.819118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:03.819118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:03.819118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:03.819118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:03.819118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:03.819118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:03.819118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:03.819118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:03.819118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:03.819118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:03.819118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:03.819118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:03.819118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:03.821517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:03.821517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:03.821517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:03.821517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:03.821517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:03.821517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:03.986347 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:03.987219 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:03.987484 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:03.989185 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:03.989597 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:03.989870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:03.990958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:03.990958 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:03.990958 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:03.990958 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:03.990958 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:03.990958 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:03.990958 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:03.990958 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:03.992227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:03.992227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:03.992227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:03.992227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:03.992227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:03.992227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:41:03.992227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:03.992227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:03.992227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:03.992227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:03.992227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:03.992227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:03.992817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:03.992817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:03.992817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:03.992817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:03.992817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:03.992817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:03.992817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:03.992817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:03.992817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:03.992817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:03.992817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:03.992817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:03.992817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:03.993339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:03.993339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:03.993339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:03.993339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:03.993339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:03.993339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:03.993339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:03.993339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:03.993339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:03.993339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:03.993339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:03.993339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:03.993339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:03.993339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:03.993339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:03.994012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:03.994012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:03.994012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:03.994012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:41:03.994012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:03.994012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:03.994012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:03.994012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:03.994012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:03.994012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:03.994012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:03.994012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:03.994012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:03.994012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:03.994012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:03.994012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:03.994012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:03.994012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:03.994012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:03.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:03.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:03.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:03.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:03.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:03.994840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:04.015181 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:04.015606 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:04.015857 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:04.016047 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:41:04.016956 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:04.017314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:04.018375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:04.018375 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:04.018375 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:04.018375 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:04.018375 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:04.018375 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:04.018375 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:04.018375 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:04.019582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:04.019582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:41:04.019582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:04.019582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:04.019582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:04.019582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:04.019582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:04.019582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:04.019582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:04.019582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:04.019582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:04.019582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:41:04.019582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:04.019582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:41:04.019582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:41:04.020519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:04.020519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:41:04.020519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:04.020519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:41:04.020519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:41:04.020519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:04.020519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:41:04.020519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:04.020519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:41:04.021146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:04.021146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:04.021146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:04.021146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:04.021146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:04.021146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:41:04.021146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:41:04.021146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:41:04.021146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:04.021146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:04.021146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:04.021146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:04.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:04.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:04.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:04.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:04.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:04.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:04.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:04.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:04.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:04.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:04.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:04.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:04.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:04.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:04.022475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:04.022475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:04.022475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:04.022475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:04.022961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:04.022961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:04.022961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:04.022961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:04.022961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:04.022961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:04.022961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:04.022961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:04.022961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:04.022961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:04.022961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:04.022961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:04.022961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:04.022961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:04.022961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:04.022961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:04.023856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:41:04.023856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:41:04.023856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:41:04.023856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:04.023856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:04.023856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:04.023856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:04.023856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:04.023856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:04.023856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:04.023856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:04.023856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:04.023856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:04.023856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:04.023856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:04.023856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:04.023856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:04.023856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:04.024417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:04.024417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:04.024417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:04.024417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:04.024417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:04.024417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:04.024417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:04.037022 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:04.037422 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:04.037608 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:04.037865 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:04.038808 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:04.039270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:04.040418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:04.040418 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:04.040418 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:04.040418 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:04.040418 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:04.040418 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:04.040418 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:04.040418 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:04.042194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:04.042194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:41:04.042194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:04.042194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:04.042194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:04.042194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:04.042194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:04.042194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:04.042194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:04.042194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:04.042194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:04.042194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:41:04.042194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:04.042194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:41:04.042194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:41:04.042941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:04.042941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:41:04.042941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:04.042941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:41:04.042941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:41:04.042941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:04.042941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:41:04.042941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:04.042941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:41:04.043971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:04.043971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:04.043971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:04.043971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:04.043971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:04.043971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:41:04.043971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:41:04.043971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:04.043971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:04.043971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:04.043971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:04.044505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:04.044505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:04.044505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:04.044505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:04.044505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:04.044505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:04.044505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:04.044505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:04.044505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:04.044505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:04.044505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:04.044505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:04.044505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:04.044505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:04.045188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:04.045188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:04.045188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:04.045188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:04.045929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:04.045929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:04.045929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:04.045929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:04.045929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:04.045929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:04.045929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:04.045929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:04.045929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:04.045929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:04.045929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:04.045929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:04.045929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:04.045929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:04.045929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:04.045929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:04.045929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:41:04.046701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:41:04.046701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:04.046701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:04.046701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:04.046701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:04.046701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:04.046701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:04.046701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:04.046701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:04.046701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:04.046701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:04.046701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:04.046701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:04.046701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:04.046701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:04.046701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:04.046701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:04.046701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:04.046701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:04.047635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:04.047635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:04.047635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:04.047635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:04.108412 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:04.109041 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:04.109272 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:04.109589 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:04.110645 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:04.111162 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:04.111421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:04.112695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:04.112695 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:04.112695 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:04.112695 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:04.112695 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:04.112695 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:04.112695 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:04.112695 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:04.114548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:04.114548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:04.114548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:04.114548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:04.114548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:04.114548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:41:04.114548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:04.114548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:04.114548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:04.114548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:04.114548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:04.114548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:04.114931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:04.114931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:04.114931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:04.114931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:04.114931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:04.114931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:04.114931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:04.114931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:04.114931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:04.114931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:04.114931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:04.114931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:04.114931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:04.115296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:04.115296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:04.116623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:04.116623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:04.116623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:04.116623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:04.116623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:04.116623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:04.116623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:04.116623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:04.116623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:04.116623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:04.116623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:04.116623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:04.116623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:04.116623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:04.116623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:04.116623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:04.117088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:41:04.117088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:04.117088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:04.117088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:04.117088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:04.117088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:04.117088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:04.117088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:04.117088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:04.117088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:04.117088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:04.117088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:04.117088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:04.117088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:04.117088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:04.117088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:04.117088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:04.117088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:04.117088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:04.117604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:04.117604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:04.117604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:08.773128 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:08.773974 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:08.773974 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:08.775216 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:08.777815 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=240 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:41:08.806960 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:08.807491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:08.808500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:08.808500 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:08.808500 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:08.808500 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:08.808500 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:08.808500 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:08.808500 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:08.808500 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:08.809315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:08.809315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:08.809315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:08.809315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:08.809315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:08.809315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:41:08.809315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:08.809315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:08.809315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:08.809315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:08.809315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:08.809315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:08.810005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:08.810005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:08.810005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:08.810005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:08.810005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:08.810005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:08.810005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:08.810005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:08.810005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:08.810005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:08.810005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:08.810005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:08.810005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:08.810652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:08.810652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:08.810652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:08.810652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:08.810652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:08.810652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:08.810652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:08.810652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:08.810652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:08.810652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:08.810652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:08.810652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:08.810652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:08.810652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:08.810652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:08.811610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:08.811610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:08.811610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:08.811610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:41:08.811610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:08.811610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:08.811610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:08.811610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:08.811610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:08.811610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:08.811610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:08.811610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:08.811610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:08.811610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:08.811610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:08.811610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:08.811610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:08.811610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:08.811610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:08.812507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:08.812507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:08.812507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:08.812507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:08.812507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:08.812507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:08.987190 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:08.987691 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:08.987927 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:08.989373 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:08.989842 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:08.990135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:08.991364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:08.991364 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:08.991364 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:08.991364 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:08.991364 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:08.991364 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:08.991364 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:08.991364 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:08.992887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:08.992887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:08.992887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:08.992887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:08.992887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:08.992887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:41:08.992887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:08.992887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:08.992887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:08.992887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:08.992887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:08.992887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:08.993502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:08.993502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:08.993502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:08.993502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:08.993502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:08.993502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:08.993502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:08.993502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:08.993502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:08.993502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:08.993502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:08.993502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:08.993502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:08.994097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:08.994097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:08.994585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:08.994585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:08.994585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:08.994585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:08.994585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:08.994585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:08.994585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:08.994585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:08.994585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:08.994585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:08.994585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:08.994585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:08.994585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:08.994585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:08.994585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:08.994585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:08.995319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:41:08.995319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:08.995319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:08.995319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:08.995319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:08.995319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:08.995319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:08.995319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:08.995319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:08.995319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:08.995319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:08.995319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:08.995319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:08.995319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:08.995319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:08.995319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:08.995319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:08.995319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:08.995319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:08.995845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:08.995845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:08.995845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:09.015832 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:09.016279 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:09.016457 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:09.016646 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:41:09.017463 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:09.017871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:09.018918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:09.018918 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:09.018918 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:09.018918 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:09.018918 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:09.018918 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:09.018918 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:09.018918 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:09.020164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:09.020164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:41:09.020164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:09.020164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:09.020164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:09.020164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:09.020164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:09.020164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:09.020164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:09.020164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:09.020164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:09.020164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:41:09.020164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:09.020164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:41:09.020164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:41:09.020948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:09.020948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:41:09.020948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:09.020948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:41:09.020948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:41:09.020948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:09.020948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:41:09.020948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:09.020948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:41:09.021464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:09.021464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:09.021464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:09.021464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:09.021464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:09.021464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:41:09.021464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:41:09.021464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:41:09.021464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:09.021464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:09.021464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:09.021464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:09.022067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:09.022067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:09.022067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:09.022067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:09.022067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:09.022067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:09.022067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:09.022067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:09.022067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:09.022067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:09.022067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:09.022067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:09.022067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:09.022067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:09.022765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:09.022765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:09.022765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:09.022765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:09.023351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:09.023351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:09.023351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:09.023351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:09.023351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:09.023351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:09.023351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:09.023351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:09.023351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:09.023351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:09.023351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:09.023351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:09.023351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:09.023351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:09.023351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:09.023351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:09.023839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:41:09.023839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:41:09.023839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:41:09.023839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:09.023839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:09.023839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:09.023839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:09.023839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:09.023839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:09.023839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:09.023839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:09.023839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:09.023839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:09.023839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:09.023839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:09.023839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:09.023839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:09.023839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:09.024311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:09.024311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:09.024311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:09.024311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:09.024311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:09.024311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:09.024311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:09.037537 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:09.037979 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:09.038158 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:09.038356 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:09.038937 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:09.039226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:09.039917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:09.039917 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:09.039917 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:09.039917 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:09.039917 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:09.039917 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:09.039917 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:09.039917 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:09.041148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:09.041148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:41:09.041148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:09.041148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:09.041148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:09.041148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:09.041148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:09.041148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:09.041148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:09.041148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:09.041148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:09.041148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:41:09.041148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:09.041148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:41:09.041148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:41:09.041906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:09.041906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:41:09.041906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:09.041906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:41:09.041906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:41:09.041906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:09.041906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:41:09.041906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:09.041906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:41:09.042620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:09.042620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:09.042620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:09.042620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:09.042620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:09.042620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:41:09.042620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:41:09.042620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:09.042620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:09.042620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:09.042620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:09.043233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:09.043233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:09.043233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:09.043233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:09.043233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:09.043233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:09.043233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:09.043233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:09.043233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:09.043233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:09.043233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:09.043233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:09.043233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:09.043233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:09.043629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:09.043629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:09.043629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:09.043629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:09.043982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:09.043982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:09.043982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:09.043982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:09.043982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:09.043982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:09.043982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:09.043982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:09.043982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:09.043982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:09.043982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:09.043982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:09.043982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:09.043982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:09.043982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:09.043982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:09.043982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:41:09.044473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:41:09.044473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:09.044473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:09.044473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:09.044473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:09.044473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:09.044473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:09.044473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:09.044473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:09.044473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:09.044473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:09.044473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:09.044473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:09.044473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:09.044473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:09.044473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:09.044473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:09.044473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:09.044473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:09.044971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:09.044971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:09.044971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:09.044971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:09.109186 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:09.109637 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:09.109877 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:09.110065 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:09.110977 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:09.111519 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:09.111768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:09.112774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:09.112774 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:09.112774 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:09.112774 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:09.112774 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:09.112774 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:09.112774 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:09.112774 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:09.114094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:09.114094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:09.114094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:09.114094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:09.114094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:09.114094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:41:09.114094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:09.114094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:09.114094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:09.114094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:09.114094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:09.114094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:09.114754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:09.114754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:09.114754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:09.114754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:09.114754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:09.114754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:09.114754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:09.114754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:09.114754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:09.114754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:09.114754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:09.114754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:09.114754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:09.115299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:09.115299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:09.115697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:09.115697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:09.115697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:09.115697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:09.115697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:09.115697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:09.115697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:09.115697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:09.115697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:09.115697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:09.115697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:09.115697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:09.115697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:09.115697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:09.115697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:09.115697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:09.116419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:41:09.116419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:09.116419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:09.116419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:09.116419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:09.116419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:09.116419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:09.116419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:09.116419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:09.116419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:09.116419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:09.116419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:09.116419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:09.116419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:09.116419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:09.116419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:09.116419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:09.116419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:09.116419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:09.117289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:09.117289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:09.117289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:13.056862 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:13.057438 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:13.057438 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:13.067354 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
Apr 28 20:41:13.067644 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=106506) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
Apr 28 20:41:13.068026 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 7ff08466-af6d-4e13-bb1d-8987170a5734 (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:41:13.096192 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-7ff08466-af6d-4e13-bb1d-8987170a5734, resource={'id': '7ff08466-af6d-4e13-bb1d-8987170a5734', 'name': 'private', 'tenant_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'project_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['4b461bd9-09ea-4f87-a007-a11ab022ff7f', '8f3677a2-e2be-443a-a90c-2cd7b3095c49'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-28T20:10:58Z', 'updated_at': '2026-04-28T20:11:02Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 46342}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:13.096481 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-7ff08466-af6d-4e13-bb1d-8987170a5734, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:13.097004 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-27fbc209-4b05-4b4a-8d5c-40d6d87b7e22, external_ids_update=None, columns={'tag': 46342}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:13.097128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.097790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:13.097790 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:13.097790 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:13.097790 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:13.097790 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:13.097790 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:13.097790 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.097790 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:13.099031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 7ff08466-af6d-4e13-bb1d-8987170a5734 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.099031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:13.099031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:41:13.099031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:41:13.099031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:41:13.099031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:41:13.099031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:41:13.099031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:13.099031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:13.099031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:13.099031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:13.099297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:13.099297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:13.099297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:13.099297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:13.099297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:13.099297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:13.099297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:13.099297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:13.099297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:13.099297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:13.099297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:13.099297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:13.099297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:13.099297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:13.099601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:13.099601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.099601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:13.099601 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 8a1fb774-5693-4139-8b75-6ab5273228c7 (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:41:13.126665 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, resource={'id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'name': 'public', 'tenant_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'project_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['335d12ce-2d0c-4086-b3f4-2af390c03d70', '5308c10f-28d1-441c-acb2-d1d6dc3d2ece'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-04-28T20:11:05Z', 'updated_at': '2026-04-28T20:11:13Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:13.127070 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:13.127635 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-7dbf713e-d963-4884-ae64-da4ce196e719, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:13.127999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.128768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:13.128768 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:13.128768 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:13.128768 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:13.128768 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:13.128768 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:13.128768 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.128768 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:13.129916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 8a1fb774-5693-4139-8b75-6ab5273228c7 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.129916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:13.129916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:41:13.129916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:41:13.129916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:41:13.129916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:41:13.129916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:41:13.129916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:13.129916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:13.129916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:13.129916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:13.130423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:13.130423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:13.130423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:13.130423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:13.130423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:13.130423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:13.130423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:13.130423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:13.130423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:13.130423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:13.130423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:13.130423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:13.130423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:13.130423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:13.131033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:13.131033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.131033 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:13.131033 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource c6718637-daa9-4db1-bd11-63abe3891db6 (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:41:13.157483 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-c6718637-daa9-4db1-bd11-63abe3891db6, resource={'id': 'c6718637-daa9-4db1-bd11-63abe3891db6', 'name': 'admin_net', 'tenant_id': '3eb96b75a3044dd0a338616e1d9081d0', 'project_id': '3eb96b75a3044dd0a338616e1d9081d0', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['c1cc9883-a7e2-4abc-8892-a7c1b200d3ad'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-28T20:15:13Z', 'updated_at': '2026-04-28T20:15:14Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 7626}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:13.157716 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-c6718637-daa9-4db1-bd11-63abe3891db6, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:13.158134 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-7461d8df-a8a0-40c3-87a1-26ed88f9f7ce, external_ids_update=None, columns={'tag': 7626}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:13.158134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.158613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:13.158613 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:13.158613 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:13.158613 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:13.158613 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:13.158613 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:13.158613 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.158613 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:13.160042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource c6718637-daa9-4db1-bd11-63abe3891db6 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.160042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:13.160042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:41:13.160042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:41:13.160042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:41:13.160042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:41:13.160042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:41:13.160042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:13.160042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:13.160042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:13.160042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:13.160392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:13.160392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:13.160392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:13.160392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:13.160392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:13.160392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:13.160392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:13.160392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:13.160392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:13.160392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:13.160392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:13.160392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:13.160392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:13.160392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:13.160881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:13.160881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.160881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:13.160881 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource d3dfebbd-7943-4813-b509-739084ada63b (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:41:13.181473 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d3dfebbd-7943-4813-b509-739084ada63b, resource={'id': 'd3dfebbd-7943-4813-b509-739084ada63b', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-pi48i', 'tenant_id': 'a11fcd8844504e7eaf509ece683e8c8f', 'project_id': 'a11fcd8844504e7eaf509ece683e8c8f', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['c68bc19b-c88a-4cd2-bf3a-d847d267d23f'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-28T20:15:49Z', 'updated_at': '2026-04-28T20:15:50Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 58143}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:13.181738 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-d3dfebbd-7943-4813-b509-739084ada63b, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-pi48i', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:13.182066 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-ed33b0a1-b0b7-4e33-aa96-92c114a82f5d, external_ids_update=None, columns={'tag': 58143}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:13.182169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.182735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:13.182735 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:13.182735 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:13.182735 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:13.182735 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:13.182735 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:13.182735 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.182735 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:13.183605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource d3dfebbd-7943-4813-b509-739084ada63b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.183605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:13.183605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:41:13.183605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:41:13.183605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:41:13.183605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:41:13.183605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:41:13.183605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:13.183605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:13.183605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:13.183605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:13.183873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:13.183873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:13.183873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:13.183873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:13.183873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:13.183873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:13.183873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:13.183873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:13.183873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:13.183873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:13.183873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:13.183873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:13.183873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:13.183873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:13.184221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:13.184221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.184221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:13.184221 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 6feb526a-13c6-41ac-983d-ea1d2251ff3d (type: routers) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:41:13.247981 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, resource={'id': '6feb526a-13c6-41ac-983d-ea1d2251ff3d', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-pi48i', 'project_id': 'a11fcd8844504e7eaf509ece683e8c8f', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'external_fixed_ips': [{'subnet_id': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece', 'ip_address': '172.24.5.87'}, {'subnet_id': '335d12ce-2d0c-4086-b3f4-2af390c03d70', 'ip_address': '2001:db8::2d8'}], 'enable_snat': True}, 'gw_port_id': '73493630-b0ac-48ec-be98-5b048d2e2ce7', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-pi48i', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'external_fixed_ips': [{'ip_address': '172.24.5.87', 'subnet_id': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece'}, {'ip_address': '2001:db8::2d8', 'subnet_id': '335d12ce-2d0c-4086-b3f4-2af390c03d70'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-04-28T20:15:51Z', 'updated_at': '2026-04-28T20:15:54Z', 'revision_number': 3, 'tenant_id': 'a11fcd8844504e7eaf509ece683e8c8f'}, resource_type=routers, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:13.248280 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-pi48i', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:13.248621 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "73493630-b0ac-48ec-be98-5b048d2e2ce7"')], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:13.248884 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "73493630-b0ac-48ec-be98-5b048d2e2ce7"')], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:13.249064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.249788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:13.249788 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:13.249788 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:13.249788 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:13.249788 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:13.249788 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:13.249788 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.249788 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:13.249950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Unable to update router 6feb526a-13c6-41ac-983d-ea1d2251ff3d. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.251043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 6feb526a-13c6-41ac-983d-ea1d2251ff3d (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.251043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:13.251043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:41:13.251043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:41:13.251043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:41:13.251043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:41:13.251043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
Apr 28 20:41:13.251043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Apr 28 20:41:13.251043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
Apr 28 20:41:13.251043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Apr 28 20:41:13.251043 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Apr 28 20:41:13.251313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Apr 28 20:41:13.251313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
Apr 28 20:41:13.251313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:13.251313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:13.251313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:13.251313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:13.251313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:13.251313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:13.251313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:13.251313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:13.251313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:13.251313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:13.251313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:13.251313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:13.251604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:13.251604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:13.251604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:13.251604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:13.251604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:13.251604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:13.251604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:13.251604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.251604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:13.251604 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 73493630-b0ac-48ec-be98-5b048d2e2ce7 (type: router_ports) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:41:13.340284 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-73493630-b0ac-48ec-be98-5b048d2e2ce7, resource={'id': '73493630-b0ac-48ec-be98-5b048d2e2ce7', 'name': '', 'network_id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'project_id': '', 'mac_address': 'fa:16:3e:f8:07:38', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6feb526a-13c6-41ac-983d-ea1d2251ff3d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece', 'ip_address': '172.24.5.87'}, {'subnet_id': '335d12ce-2d0c-4086-b3f4-2af390c03d70', 'ip_address': '2001:db8::2d8'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000171185', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-04-28T20:15:51Z', 'updated_at': '2026-04-28T20:15:54Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:13.340529 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-73493630-b0ac-48ec-be98-5b048d2e2ce7, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece 335d12ce-2d0c-4086-b3f4-2af390c03d70', 'neutron:network_name': 'neutron-8a1fb774-5693-4139-8b75-6ab5273228c7', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.87/24', '2001:db8::2d8/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:13.341070 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=73493630-b0ac-48ec-be98-5b048d2e2ce7, lrouter_port=lrp-73493630-b0ac-48ec-be98-5b048d2e2ce7, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:13.341416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.342182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:13.342182 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:13.342182 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:13.342182 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:13.342182 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:13.342182 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:13.342182 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.342182 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:13.343231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 73493630-b0ac-48ec-be98-5b048d2e2ce7 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.343231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:13.343231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:41:13.343231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:41:13.343231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:41:13.343231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:41:13.343231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1971, in update_router_port
Apr 28 20:41:13.343231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:13.343231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:13.343231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:13.343231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:13.343787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:13.343787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:13.343787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:13.343787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:13.343787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:13.343787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:13.343787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:13.343787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:13.343787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:13.343787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:13.343787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:13.343787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:13.343787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:13.343787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:13.344442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:13.344442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.344442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:13.344442 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_for_inconsistencies finished in 0.286 seconds
Apr 28 20:41:13.773861 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:13.774478 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:13.774707 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:13.775924 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:13.812979 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:13.813335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.814130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:13.814130 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:13.814130 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:13.814130 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:13.814130 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:13.814130 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:13.814130 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.814130 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:13.815171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.815171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:13.815171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:13.815171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:13.815171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:13.815171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:41:13.815171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:13.815171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:13.815171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:13.815171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:13.815171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:13.815171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:13.815755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:13.815755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:13.815755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:13.815755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:13.815755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:13.815755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:13.815755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:13.815755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:13.815755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:13.815755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:13.815755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:13.815755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:13.815755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:13.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:13.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:13.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:13.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:13.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:13.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:13.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:13.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:13.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:13.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:13.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:13.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:13.816303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:13.817111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:13.817111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:13.817111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:13.817111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:41:13.817111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:13.817111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:13.817111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:13.817111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:13.817111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:13.817111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:13.817111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:13.817111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:13.817111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:13.817111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:13.817111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:13.817111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:13.817111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:13.817111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:13.817111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:13.817788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:13.817788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:13.817788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:13.817788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:13.817788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.817788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:13.987952 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:13.988591 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:13.988812 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:13.989757 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:13.990062 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:13.990286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.991058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:13.991058 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:13.991058 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:13.991058 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:13.991058 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:13.991058 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:13.991058 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.991058 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:13.992540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.992540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:13.992540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:13.992540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:13.992540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:13.992540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:41:13.992540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:13.992540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:13.992540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:13.992540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:13.992540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:13.992540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:13.993218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:13.993218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:13.993218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:13.993218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:13.993218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:13.993218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:13.993218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:13.993218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:13.993218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:13.993218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:13.993218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:13.993218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:13.993218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:13.993566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.993566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:13.994239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.994239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:13.994239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:13.994239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:13.994239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:13.994239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:13.994239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:13.994239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:13.994239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:13.994239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:13.994239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:13.994239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:13.994239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:13.994239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:13.994239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:13.994239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:13.994757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:41:13.994757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:13.994757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:13.994757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:13.994757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:13.994757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:13.994757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:13.994757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:13.994757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:13.994757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:13.994757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:13.994757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:13.994757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:13.994757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:13.994757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:13.994757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:13.994757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:13.994757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:13.994757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:13.995255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:13.995255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:13.995255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:14.016344 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:14.016705 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:14.016970 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:14.017231 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:41:14.017702 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:14.017989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:14.018806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:14.018806 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:14.018806 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:14.018806 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:14.018806 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:14.018806 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:14.018806 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:14.018806 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:14.020052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:14.020052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:41:14.020052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:14.020052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:14.020052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:14.020052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:14.020052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:14.020052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:14.020052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:14.020052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:14.020052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:14.020052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:41:14.020052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:14.020052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:41:14.020052 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:41:14.020838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:14.020838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:41:14.020838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:14.020838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:41:14.020838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:41:14.020838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:14.020838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:41:14.020838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:14.020838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:41:14.021394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:14.021394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:14.021394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:14.021394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:14.021394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:14.021394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:41:14.021394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:41:14.021394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:41:14.021394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:14.021394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:14.021394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:14.021394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:14.022002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:14.022002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:14.022002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:14.022002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:14.022002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:14.022002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:14.022002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:14.022002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:14.022002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:14.022002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:14.022002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:14.022002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:14.022002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:14.022002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:14.022637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:14.022637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:14.022637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:14.022637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:14.023075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:14.023075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:14.023075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:14.023075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:14.023075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:14.023075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:14.023075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:14.023075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:14.023075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:14.023075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:14.023075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:14.023075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:14.023075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:14.023075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:14.023075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:14.023075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:14.023755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:41:14.023755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:41:14.023755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:41:14.023755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:14.023755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:14.023755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:14.023755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:14.023755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:14.023755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:14.023755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:14.023755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:14.023755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:14.023755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:14.023755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:14.023755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:14.023755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:14.023755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:14.023755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:14.024428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:14.024428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:14.024428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:14.024428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:14.024428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:14.024428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:14.024428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:14.037980 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:14.038430 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:14.038618 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:14.038867 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:14.039386 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:14.039731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:14.040434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:14.040434 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:14.040434 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:14.040434 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:14.040434 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:14.040434 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:14.040434 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:14.040434 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:14.041797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:14.041797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:41:14.041797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:14.041797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:14.041797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:14.041797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:14.041797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:14.041797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:14.041797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:14.041797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:14.041797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:14.041797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:41:14.041797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:14.041797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:41:14.041797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:41:14.042450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:14.042450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:41:14.042450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:14.042450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:41:14.042450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:41:14.042450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:14.042450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:41:14.042450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:14.042450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:41:14.043197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:14.043197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:14.043197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:14.043197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:14.043197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:14.043197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:41:14.043197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:41:14.043197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:14.043197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:14.043197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:14.043197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:14.043764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:14.043764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:14.043764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:14.043764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:14.043764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:14.043764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:14.043764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:14.043764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:14.043764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:14.043764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:14.043764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:14.043764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:14.043764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:14.043764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:14.044366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:14.044366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:14.044366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:14.044366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:14.044937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:14.044937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:14.044937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:14.044937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:14.044937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:14.044937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:14.044937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:14.044937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:14.044937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:14.044937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:14.044937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:14.044937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:14.044937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:14.044937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:14.044937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:14.044937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:14.044937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:41:14.045760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:41:14.045760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:14.045760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:14.045760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:14.045760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:14.045760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:14.045760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:14.045760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:14.045760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:14.045760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:14.045760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:14.045760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:14.045760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:14.045760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:14.045760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:14.045760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:14.045760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:14.045760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:14.045760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:14.046569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:14.046569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:14.046569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:14.046569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:14.109747 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:14.110448 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:14.110448 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:14.110575 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:14.111055 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:14.111420 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:14.111653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:14.112367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:14.112367 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:14.112367 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:14.112367 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:14.112367 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:14.112367 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:14.112367 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:14.112367 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:14.113715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:14.113715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:14.113715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:14.113715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:14.113715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:14.113715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:41:14.113715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:14.113715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:14.113715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:14.113715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:14.113715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:14.113715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:14.114279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:14.114279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:14.114279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:14.114279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:14.114279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:14.114279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:14.114279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:14.114279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:14.114279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:14.114279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:14.114279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:14.114279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:14.114279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:14.114852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:14.114852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:14.115222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:14.115222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:14.115222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:14.115222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:14.115222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:14.115222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:14.115222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:14.115222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:14.115222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:14.115222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:14.115222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:14.115222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:14.115222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:14.115222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:14.115222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:14.115222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:14.115948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:41:14.115948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:14.115948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:14.115948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:14.115948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:14.115948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:14.115948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:14.115948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:14.115948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:14.115948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:14.115948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:14.115948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:14.115948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:14.115948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:14.115948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:14.115948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:14.115948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:14.115948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:14.115948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:14.116756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:14.116756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:14.116756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:18.774610 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:18.775596 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:18.775596 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:18.776122 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:18.810839 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:18.811094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:18.811922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:18.811922 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:18.811922 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:18.811922 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:18.811922 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:18.811922 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:18.811922 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:18.811922 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:18.812947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:18.812947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:18.812947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:18.812947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:18.812947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:18.812947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:41:18.812947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:18.812947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:18.812947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:18.812947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:18.812947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:18.812947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:18.813603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:18.813603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:18.813603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:18.813603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:18.813603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:18.813603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:18.813603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:18.813603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:18.813603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:18.813603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:18.813603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:18.813603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:18.813603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:18.814246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:18.814246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:18.814246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:18.814246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:18.814246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:18.814246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:18.814246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:18.814246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:18.814246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:18.814246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:18.814246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:18.814246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:18.814246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:18.814246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:18.814246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:18.814986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:18.814986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:18.814986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:18.814986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:41:18.814986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:18.814986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:18.814986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:18.814986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:18.814986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:18.814986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:18.814986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:18.814986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:18.814986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:18.814986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:18.814986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:18.814986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:18.814986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:18.814986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:18.814986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:18.816012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:18.816012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:18.816012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:18.816012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:18.816012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:18.816012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:18.988794 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:18.989326 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:18.989516 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:18.990654 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:18.991012 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:18.991237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:18.992258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:18.992258 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:18.992258 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:18.992258 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:18.992258 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:18.992258 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:18.992258 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:18.992258 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:18.993924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:18.993924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:18.993924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:18.993924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:18.993924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:18.993924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:41:18.993924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:18.993924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:18.993924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:18.993924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:18.993924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:18.993924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:18.994542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:18.994542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:18.994542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:18.994542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:18.994542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:18.994542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:18.994542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:18.994542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:18.994542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:18.994542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:18.994542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:18.994542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:18.994542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:18.995137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:18.995137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:18.995556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:18.995556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:18.995556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:18.995556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:18.995556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:18.995556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:18.995556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:18.995556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:18.995556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:18.995556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:18.995556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:18.995556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:18.995556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:18.995556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:18.995556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:18.995556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:18.996502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:41:18.996502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:18.996502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:18.996502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:18.996502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:18.996502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:18.996502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:18.996502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:18.996502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:18.996502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:18.996502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:18.996502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:18.996502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:18.996502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:18.996502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:18.996502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:18.996502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:18.996502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:18.996502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:18.997417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:18.997417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:18.997417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:19.017021 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:19.017546 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:19.017863 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:19.018135 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:41:19.018848 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:19.019211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:19.020137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:19.020137 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:19.020137 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:19.020137 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:19.020137 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:19.020137 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:19.020137 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:19.020137 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:19.021496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:19.021496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:41:19.021496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:19.021496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:19.021496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:19.021496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:19.021496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:19.021496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:19.021496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:19.021496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:19.021496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:19.021496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:41:19.021496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:19.021496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:41:19.021496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:41:19.022026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:19.022026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:41:19.022026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:19.022026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:41:19.022026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:41:19.022026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:19.022026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:41:19.022026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:19.022026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:41:19.022980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:19.022980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:19.022980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:19.022980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:19.022980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:19.022980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:41:19.022980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:41:19.022980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:41:19.022980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:19.022980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:19.022980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:19.022980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:19.023404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:19.023404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:19.023404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:19.023404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:19.023404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:19.023404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:19.023404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:19.023404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:19.023404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:19.023404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:19.023404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:19.023404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:19.023404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:19.023404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:19.023862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:19.023862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:19.023862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:19.023862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:19.025072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:19.025072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:19.025072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:19.025072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:19.025072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:19.025072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:19.025072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:19.025072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:19.025072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:19.025072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:19.025072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:19.025072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:19.025072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:19.025072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:19.025072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:19.025072 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:19.025640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:41:19.025640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:41:19.025640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:41:19.025640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:19.025640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:19.025640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:19.025640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:19.025640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:19.025640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:19.025640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:19.025640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:19.025640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:19.025640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:19.025640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:19.025640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:19.025640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:19.025640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:19.025640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:19.026237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:19.026237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:19.026237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:19.026237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:19.026237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:19.026237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:19.026237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:19.038463 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:19.039053 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:19.039251 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:19.039465 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:19.040155 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:19.040438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:19.041046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:19.041046 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:19.041046 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:19.041046 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:19.041046 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:19.041046 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:19.041046 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:19.041046 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:19.042321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:19.042321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:41:19.042321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:19.042321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:19.042321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:19.042321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:19.042321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:19.042321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:19.042321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:19.042321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:19.042321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:19.042321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:41:19.042321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:19.042321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:41:19.042321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:41:19.043165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:19.043165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:41:19.043165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:19.043165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:41:19.043165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:41:19.043165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:19.043165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:41:19.043165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:19.043165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:41:19.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:19.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:19.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:19.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:19.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:19.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:41:19.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:41:19.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:19.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:19.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:19.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:19.044525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:19.044525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:19.044525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:19.044525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:19.044525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:19.044525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:19.044525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:19.044525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:19.044525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:19.044525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:19.044525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:19.044525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:19.044525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:19.044525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:19.044982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:19.044982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:19.044982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:19.044982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:19.045688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:19.045688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:19.045688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:19.045688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:19.045688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:19.045688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:19.045688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:19.045688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:19.045688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:19.045688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:19.045688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:19.045688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:19.045688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:19.045688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:19.045688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:19.045688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:19.045688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:41:19.046166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:41:19.046166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:19.046166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:19.046166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:19.046166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:19.046166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:19.046166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:19.046166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:19.046166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:19.046166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:19.046166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:19.046166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:19.046166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:19.046166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:19.046166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:19.046166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:19.046166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:19.046166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:19.046166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:19.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:19.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:19.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:19.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:19.110343 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:19.110851 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:19.111040 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:19.111304 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:19.111940 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:19.112310 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:19.112538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:19.113758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:19.113758 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:19.113758 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:19.113758 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:19.113758 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:19.113758 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:19.113758 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:19.113758 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:19.115219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:19.115219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:19.115219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:19.115219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:19.115219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:19.115219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:41:19.115219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:19.115219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:19.115219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:19.115219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:19.115219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:19.115219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:19.115949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:19.115949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:19.115949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:19.115949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:19.115949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:19.115949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:19.115949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:19.115949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:19.115949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:19.115949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:19.115949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:19.115949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:19.115949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:19.116498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:19.116498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:19.116968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:19.116968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:19.116968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:19.116968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:19.116968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:19.116968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:19.116968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:19.116968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:19.116968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:19.116968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:19.116968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:19.116968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:19.116968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:19.116968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:19.116968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:19.116968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:19.117648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:41:19.117648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:19.117648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:19.117648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:19.117648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:19.117648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:19.117648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:19.117648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:19.117648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:19.117648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:19.117648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:19.117648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:19.117648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:19.117648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:19.117648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:19.117648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:19.117648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:19.117648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:19.117648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:19.118210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:19.118210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:19.118210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:23.775437 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:23.776042 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:23.776255 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:23.777547 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:23.810117 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:23.810578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:23.811759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:23.811759 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:23.811759 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:23.811759 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:23.811759 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:23.811759 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:23.811759 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:23.811759 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:23.812757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:23.812757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:23.812757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:23.812757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:23.812757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:23.812757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:41:23.812757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:23.812757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:23.812757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:23.812757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:23.812757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:23.812757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:23.813385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:23.813385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:23.813385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:23.813385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:23.813385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:23.813385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:23.813385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:23.813385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:23.813385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:23.813385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:23.813385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:23.813385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:23.813385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:23.814024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:23.814024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:23.814024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:23.814024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:23.814024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:23.814024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:23.814024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:23.814024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:23.814024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:23.814024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:23.814024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:23.814024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:23.814024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:23.814024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:23.814024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:23.814740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:23.814740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:23.814740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:23.814740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:41:23.814740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:23.814740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:23.814740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:23.814740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:23.814740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:23.814740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:23.814740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:23.814740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:23.814740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:23.814740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:23.814740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:23.814740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:23.814740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:23.814740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:23.814740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:23.815592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:23.815592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:23.815592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:23.815592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:23.815592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:23.815592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:23.989297 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:23.989975 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:23.990300 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:23.991282 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:23.991464 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:23.991591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:23.992219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:23.992219 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:23.992219 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:23.992219 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:23.992219 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:23.992219 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:23.992219 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:23.992219 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:23.994265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:23.994265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:23.994265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:23.994265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:23.994265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:23.994265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:41:23.994265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:23.994265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:23.994265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:23.994265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:23.994265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:23.994265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:23.994621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:23.994621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:23.994621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:23.994621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:23.994621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:23.994621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:23.994621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:23.994621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:23.994621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:23.994621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:23.994621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:23.994621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:23.994621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:23.995044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:23.995044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:23.996117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:23.996117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:23.996117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:23.996117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:23.996117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:23.996117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:23.996117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:23.996117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:23.996117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:23.996117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:23.996117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:23.996117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:23.996117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:23.996117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:23.996117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:23.996117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:23.996526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:41:23.996526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:23.996526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:23.996526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:23.996526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:23.996526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:23.996526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:23.996526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:23.996526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:23.996526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:23.996526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:23.996526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:23.996526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:23.996526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:23.996526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:23.996526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:23.996526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:23.996526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:23.996526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:23.997019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:23.997019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:23.997019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:24.017450 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:24.017887 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:24.018081 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:24.018374 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:41:24.019017 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:24.019335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:24.020214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:24.020214 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:24.020214 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:24.020214 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:24.020214 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:24.020214 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:24.020214 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:24.020214 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:24.021421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:24.021421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:41:24.021421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:24.021421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:24.021421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:24.021421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:24.021421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:24.021421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:24.021421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:24.021421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:24.021421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:24.021421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:41:24.021421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:24.021421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:41:24.021421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:41:24.022055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:24.022055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:41:24.022055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:24.022055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:41:24.022055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:41:24.022055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:24.022055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:41:24.022055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:24.022055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:41:24.022942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:24.022942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:24.022942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:24.022942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:24.022942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:24.022942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:41:24.022942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:41:24.022942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:41:24.022942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:24.022942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:24.022942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:24.022942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:24.023489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:24.023489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:24.023489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:24.023489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:24.023489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:24.023489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:24.023489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:24.023489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:24.023489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:24.023489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:24.023489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:24.023489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:24.023489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:24.023489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:24.024131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:24.024131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:24.024131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:24.024131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:24.024656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:24.024656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:24.024656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:24.024656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:24.024656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:24.024656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:24.024656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:24.024656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:24.024656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:24.024656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:24.024656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:24.024656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:24.024656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:24.024656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:24.024656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:24.024656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:24.025342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:41:24.025342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:41:24.025342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:41:24.025342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:24.025342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:24.025342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:24.025342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:24.025342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:24.025342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:24.025342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:24.025342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:24.025342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:24.025342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:24.025342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:24.025342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:24.025342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:24.025342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:24.025342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:24.026227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:24.026227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:24.026227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:24.026227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:24.026227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:24.026227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:24.026227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:24.038860 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:24.039279 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:24.039479 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:24.039701 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:24.040179 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:24.040525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:24.041306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:24.041306 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:24.041306 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:24.041306 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:24.041306 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:24.041306 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:24.041306 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:24.041306 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:24.042896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:24.042896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:41:24.042896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:24.042896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:24.042896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:24.042896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:24.042896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:24.042896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:24.042896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:24.042896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:24.042896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:24.042896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:41:24.042896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:24.042896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:41:24.042896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:41:24.043530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:24.043530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:41:24.043530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:24.043530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:41:24.043530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:41:24.043530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:24.043530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:41:24.043530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:24.043530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:41:24.044391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:24.044391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:24.044391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:24.044391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:24.044391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:24.044391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:41:24.044391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:41:24.044391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:24.044391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:24.044391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:24.044391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:24.044890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:24.044890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:24.044890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:24.044890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:24.044890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:24.044890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:24.044890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:24.044890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:24.044890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:24.044890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:24.044890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:24.044890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:24.044890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:24.044890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:24.045462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:24.045462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:24.045462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:24.045462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:24.046179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:24.046179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:24.046179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:24.046179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:24.046179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:24.046179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:24.046179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:24.046179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:24.046179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:24.046179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:24.046179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:24.046179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:24.046179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:24.046179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:24.046179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:24.046179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:24.046179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:41:24.046896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:41:24.046896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:24.046896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:24.046896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:24.046896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:24.046896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:24.046896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:24.046896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:24.046896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:24.046896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:24.046896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:24.046896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:24.046896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:24.046896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:24.046896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:24.046896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:24.046896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:24.046896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:24.046896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:24.047368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:24.047368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:24.047368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:24.047368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:24.110844 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:24.111266 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:24.111500 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:24.111843 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:24.112606 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:24.113077 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:24.113348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:24.114607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:24.114607 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:24.114607 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:24.114607 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:24.114607 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:24.114607 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:24.114607 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:24.114607 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:24.116008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:24.116008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:24.116008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:24.116008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:24.116008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:24.116008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:41:24.116008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:24.116008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:24.116008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:24.116008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:24.116008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:24.116008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:24.116522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:24.116522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:24.116522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:24.116522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:24.116522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:24.116522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:24.116522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:24.116522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:24.116522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:24.116522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:24.116522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:24.116522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:24.116522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:24.117112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:24.117112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:24.117663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:24.117663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:24.117663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:24.117663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:24.117663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:24.117663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:24.117663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:24.117663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:24.117663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:24.117663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:24.117663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:24.117663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:24.117663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:24.117663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:24.117663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:24.117663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:24.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:41:24.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:24.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:24.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:24.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:24.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:24.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:24.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:24.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:24.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:24.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:24.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:24.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:24.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:24.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:24.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:24.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:24.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:24.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:24.119279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:24.119279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:24.119279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:28.776169 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:28.777349 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:28.777349 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:28.778354 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:28.818804 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:28.818804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:28.819923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:28.819923 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:28.819923 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:28.819923 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:28.819923 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:28.819923 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:28.819923 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:28.819923 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:28.820549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:28.820549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:28.820549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:28.820549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:28.820549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:28.820549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:41:28.820549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:28.820549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:28.820549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:28.820549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:28.820549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:28.820549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:28.821221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:28.821221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:28.821221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:28.821221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:28.821221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:28.821221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:28.821221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:28.821221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:28.821221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:28.821221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:28.821221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:28.821221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:28.821221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:28.821885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:28.821885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:28.821885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:28.821885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:28.821885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:28.821885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:28.821885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:28.821885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:28.821885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:28.821885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:28.821885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:28.821885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:28.821885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:28.821885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:28.821885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:28.822642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:28.822642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:28.822642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:28.822642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:41:28.822642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:28.822642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:28.822642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:28.822642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:28.822642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:28.822642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:28.822642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:28.822642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:28.822642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:28.822642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:28.822642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:28.822642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:28.822642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:28.822642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:28.822642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:28.823603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:28.823603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:28.823603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:28.823603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:28.823603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:28.823603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:28.990073 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:28.990617 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:28.990847 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:28.992413 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:28.993113 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:28.993421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:28.994595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:28.994595 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:28.994595 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:28.994595 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:28.994595 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:28.994595 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:28.994595 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:28.994595 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:28.996228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:28.996228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:28.996228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:28.996228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:28.996228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:28.996228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:41:28.996228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:28.996228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:28.996228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:28.996228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:28.996228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:28.996228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:28.996929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:28.996929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:28.996929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:28.996929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:28.996929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:28.996929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:28.996929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:28.996929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:28.996929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:28.996929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:28.996929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:28.996929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:28.996929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:28.997624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:28.997624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:28.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:28.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:28.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:28.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:28.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:28.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:28.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:28.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:28.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:28.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:28.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:28.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:28.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:28.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:28.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:28.998163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:28.998867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:41:28.998867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:28.998867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:28.998867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:28.998867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:28.998867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:28.998867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:28.998867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:28.998867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:28.998867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:28.998867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:28.998867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:28.998867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:28.998867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:28.998867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:28.998867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:28.998867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:28.998867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:28.998867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:28.999406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:28.999406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:28.999406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:29.018156 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:29.018615 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:29.018839 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:29.019031 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:41:29.019479 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:29.019712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:29.020374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:29.020374 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:29.020374 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:29.020374 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:29.020374 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:29.020374 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:29.020374 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:29.020374 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:29.021699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:29.021699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:41:29.021699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:29.021699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:29.021699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:29.021699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:29.021699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:29.021699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:29.021699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:29.021699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:29.021699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:29.021699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:41:29.021699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:29.021699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:41:29.021699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:41:29.022504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:29.022504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:41:29.022504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:29.022504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:41:29.022504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:41:29.022504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:29.022504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:41:29.022504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:29.022504 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:41:29.023366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:29.023366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:29.023366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:29.023366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:29.023366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:29.023366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:41:29.023366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:41:29.023366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:41:29.023366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:29.023366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:29.023366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:29.023366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:29.024422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:29.024422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:29.024422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:29.024422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:29.024422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:29.024422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:29.024422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:29.024422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:29.024422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:29.024422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:29.024422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:29.024422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:29.024422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:29.024422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:29.025600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:29.025600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:29.025600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:29.025600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:29.025600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:29.025600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:29.025600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:29.025600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:29.025600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:29.025600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:29.025600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:29.025600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:29.025600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:29.025600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:29.026393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:29.026393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:29.026393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:29.026393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:29.026393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:29.026393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:29.026393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:41:29.026393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:41:29.026393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:41:29.026393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:29.026393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:29.026393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:29.026393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:29.026393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:29.026393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:29.026393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:29.026393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:29.026393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:29.026393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:29.027385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:29.027385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:29.027385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:29.027385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:29.027385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:29.027385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:29.027385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:29.027385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:29.027385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:29.027385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:29.027385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:29.027385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:29.039463 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:29.039934 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:29.040123 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:29.040333 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:29.040780 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:29.041049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:29.041662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:29.041662 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:29.041662 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:29.041662 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:29.041662 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:29.041662 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:29.041662 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:29.041662 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:29.042992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:29.042992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:41:29.042992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:29.042992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:29.042992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:29.042992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:29.042992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:29.042992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:29.042992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:29.042992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:29.042992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:29.042992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:41:29.042992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:29.042992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:41:29.042992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:41:29.043870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:29.043870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:41:29.043870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:29.043870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:41:29.043870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:41:29.043870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:29.043870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:41:29.043870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:29.043870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:41:29.044491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:29.044491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:29.044491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:29.044491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:29.044491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:29.044491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:41:29.044491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:41:29.044491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:29.044491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:29.044491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:29.044491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:29.045160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:29.045160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:29.045160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:29.045160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:29.045160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:29.045160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:29.045160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:29.045160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:29.045160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:29.045160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:29.045160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:29.045160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:29.045160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:29.045160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:29.045834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:29.045834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:29.045834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:29.045834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:29.046392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:29.046392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:29.046392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:29.046392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:29.046392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:29.046392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:29.046392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:29.046392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:29.046392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:29.046392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:29.046392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:29.046392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:29.046392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:29.046392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:29.046392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:29.046392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:29.046392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:41:29.046903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:41:29.046903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:29.046903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:29.046903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:29.046903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:29.046903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:29.046903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:29.046903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:29.046903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:29.046903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:29.046903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:29.046903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:29.046903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:29.046903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:29.046903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:29.046903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:29.046903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:29.046903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:29.046903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:29.047460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:29.047460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:29.047460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:29.047460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:29.111611 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:29.112361 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:29.112469 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:29.112842 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:29.113605 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:29.114207 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:29.114445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:29.116491 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:29.116491 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:29.116491 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:29.116491 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:29.116491 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:29.116491 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:29.116491 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:29.116491 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:29.118342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:29.118342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:29.118342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:29.118342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:29.118342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:29.118342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:41:29.118342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:29.118342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:29.118342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:29.118342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:29.118342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:29.118342 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:29.119171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:29.119171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:29.119171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:29.119171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:29.119171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:29.119171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:29.119171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:29.119171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:29.119171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:29.119171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:29.119171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:29.119171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:29.119171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:29.119809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:29.119809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:29.119945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:29.119945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:29.119945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:29.119945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:29.119945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:29.119945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:29.119945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:29.119945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:29.119945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:29.119945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:29.119945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:29.119945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:29.119945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:29.119945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:29.119945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:29.119945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:29.120643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:41:29.120643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:29.120643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:29.120643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:29.120643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:29.120643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:29.120643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:29.120643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:29.120643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:29.120643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:29.120643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:29.120643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:29.120643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:29.120643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:29.120643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:29.120643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:29.120643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:29.120643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:29.120643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:29.121534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:29.121534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:29.121534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:33.777013 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:33.777848 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:33.777848 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:33.779142 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:33.811023 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:33.811584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:33.812893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:33.812893 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:33.812893 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:33.812893 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:33.812893 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:33.812893 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:33.812893 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:33.812893 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:33.814005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:33.814005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:33.814005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:33.814005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:33.814005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:33.814005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:41:33.814005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:33.814005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:33.814005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:33.814005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:33.814005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:33.814005 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:33.814470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:33.814470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:33.814470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:33.814470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:33.814470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:33.814470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:33.814470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:33.814470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:33.814470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:33.814470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:33.814470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:33.814470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:33.814470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:33.814952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:33.814952 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:33.815509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:33.815509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:33.815509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:33.815509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:33.815509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:33.815509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:33.815509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:33.815509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:33.815509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:33.815509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:33.815509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:33.815509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:33.815509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:33.815509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:33.815509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:33.815509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:33.815987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:41:33.815987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:33.815987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:33.815987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:33.815987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:33.815987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:33.815987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:33.815987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:33.815987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:33.815987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:33.815987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:33.815987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:33.815987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:33.815987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:33.815987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:33.815987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:33.815987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:33.815987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:33.815987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:33.816562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:33.816562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:33.816562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:33.991093 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:33.991723 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:33.991922 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:33.993592 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:33.994044 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:33.994322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:33.995532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:33.995532 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:33.995532 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:33.995532 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:33.995532 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:33.995532 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:33.995532 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:33.995532 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:33.997449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:33.997449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:33.997449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:33.997449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:33.997449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:33.997449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:41:33.997449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:33.997449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:33.997449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:33.997449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:33.997449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:33.997449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:33.998230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:33.998230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:33.998230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:33.998230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:33.998230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:33.998230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:33.998230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:33.998230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:33.998230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:33.998230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:33.998230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:33.998230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:33.998230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:33.998820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:33.998820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:33.999356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:33.999356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:33.999356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:33.999356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:33.999356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:33.999356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:33.999356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:33.999356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:33.999356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:33.999356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:33.999356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:33.999356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:33.999356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:33.999356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:33.999356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:33.999356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:34.000084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:41:34.000084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:34.000084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:34.000084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:34.000084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:34.000084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:34.000084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:34.000084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:34.000084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:34.000084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:34.000084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:34.000084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:34.000084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:34.000084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:34.000084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:34.000084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:34.000084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:34.000084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:34.000084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:34.000887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:34.000887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:34.000887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:34.018721 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:34.019120 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:34.019299 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:34.019519 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:41:34.020597 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:34.021073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:34.021943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:34.021943 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:34.021943 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:34.021943 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:34.021943 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:34.021943 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:34.021943 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:34.021943 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:34.023141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:34.023141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:41:34.023141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:34.023141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:34.023141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:34.023141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:34.023141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:34.023141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:34.023141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:34.023141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:34.023141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:34.023141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:41:34.023141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:34.023141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:41:34.023141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:41:34.023925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:34.023925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:41:34.023925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:34.023925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:41:34.023925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:41:34.023925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:34.023925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:41:34.023925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:34.023925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:41:34.024479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:34.024479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:34.024479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:34.024479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:34.024479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:34.024479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:41:34.024479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:41:34.024479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:41:34.024479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:34.024479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:34.024479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:34.024479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:34.025058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:34.025058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:34.025058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:34.025058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:34.025058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:34.025058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:34.025058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:34.025058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:34.025058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:34.025058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:34.025058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:34.025058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:34.025058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:34.025058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:34.025668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:34.025668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:34.025668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:34.025668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:34.026122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:34.026122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:34.026122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:34.026122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:34.026122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:34.026122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:34.026122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:34.026122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:34.026122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:34.026122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:34.026122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:34.026122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:34.026122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:34.026122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:34.026122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:34.026122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:34.026855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:41:34.026855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:41:34.026855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:41:34.026855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:34.026855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:34.026855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:34.026855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:34.026855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:34.026855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:34.026855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:34.026855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:34.026855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:34.026855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:34.026855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:34.026855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:34.026855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:34.026855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:34.026855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:34.027760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:34.027760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:34.027760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:34.027760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:34.027760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:34.027760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:34.027760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:34.040472 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:34.040472 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:34.040698 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:34.040933 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:34.041877 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:34.046007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:34.047113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:34.047113 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:34.047113 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:34.047113 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:34.047113 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:34.047113 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:34.047113 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:34.047113 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:34.049524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:34.049524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:41:34.049524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:34.049524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:34.049524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:34.049524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:34.049524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:34.049524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:34.049524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:34.049524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:34.049524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:34.049524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:41:34.049524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:34.049524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:41:34.049524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:41:34.049954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:34.049954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:41:34.049954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:34.049954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:41:34.049954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:41:34.049954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:34.049954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:41:34.049954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:34.049954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:41:34.051325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.009 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:34.051325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:34.051325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:34.051325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:34.051325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:34.051325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:41:34.051325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:41:34.051325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:34.051325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:34.051325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:34.051325 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:34.051702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:34.051702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:34.051702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:34.051702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:34.051702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:34.051702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:34.051702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:34.051702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:34.051702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:34.051702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:34.051702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:34.051702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:34.051702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:34.051702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:34.052105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:34.052105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:34.052105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:34.052105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:34.052799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:34.052799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:34.052799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:34.052799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:34.052799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:34.052799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:34.052799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:34.052799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:34.052799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:34.052799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:34.052799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:34.052799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:34.052799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:34.052799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:34.052799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:34.052799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:34.052799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:41:34.053305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:41:34.053305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:34.053305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:34.053305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:34.053305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:34.053305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:34.053305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:34.053305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:34.053305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:34.053305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:34.053305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:34.053305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:34.053305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:34.053305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:34.053305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:34.053305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:34.053305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:34.053305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:34.053305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:34.053875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:34.053875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:34.053875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:34.053875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:34.112222 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:34.112887 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:34.113109 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:34.113388 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:34.114454 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:34.115153 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:34.115500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:34.116611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:34.116611 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:34.116611 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:34.116611 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:34.116611 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:34.116611 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:34.116611 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:34.116611 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:34.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:34.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:34.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:34.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:34.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:34.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:41:34.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:34.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:34.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:34.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:34.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:34.118370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:34.118938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:34.118938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:34.118938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:34.118938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:34.118938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:34.118938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:34.118938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:34.118938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:34.118938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:34.118938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:34.118938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:34.118938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:34.118938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:34.119577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:34.119577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:34.120215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:34.120215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:34.120215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:34.120215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:34.120215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:34.120215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:34.120215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:34.120215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:34.120215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:34.120215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:34.120215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:34.120215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:34.120215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:34.120215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:34.120215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:34.120215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:34.120905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:41:34.120905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:34.120905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:34.120905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:34.120905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:34.120905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:34.120905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:34.120905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:34.120905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:34.120905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:34.120905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:34.120905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:34.120905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:34.120905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:34.120905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:34.120905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:34.120905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:34.120905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:34.120905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:34.121696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:34.121696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:34.121696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:38.778016 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:38.778809 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:38.778809 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:38.779956 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:38.817369 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:38.817942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:38.819639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:38.819639 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:38.819639 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:38.819639 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:38.819639 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:38.819639 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:38.819639 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:38.819639 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:38.820124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:38.820124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:38.820124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:38.820124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:38.820124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:38.820124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:41:38.820124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:38.820124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:38.820124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:38.820124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:38.820124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:38.820124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:38.820703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:38.820703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:38.820703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:38.820703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:38.820703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:38.820703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:38.820703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:38.820703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:38.820703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:38.820703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:38.820703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:38.820703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:38.820703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:38.821135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:38.821135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:38.821135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:38.821135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:38.821135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:38.821135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:38.821135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:38.821135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:38.821135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:38.821135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:38.821135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:38.821135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:38.821135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:38.821135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:38.821135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:38.821891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:38.821891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:38.821891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:38.821891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:41:38.821891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:38.821891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:38.821891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:38.821891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:38.821891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:38.821891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:38.821891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:38.821891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:38.821891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:38.821891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:38.821891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:38.821891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:38.821891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:38.821891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:38.821891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:38.822913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:38.822913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:38.822913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:38.822913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:38.822913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:38.822913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:38.991756 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:38.992312 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:38.992403 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:38.993860 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:38.994270 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:38.994472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:38.995313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:38.995313 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:38.995313 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:38.995313 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:38.995313 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:38.995313 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:38.995313 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:38.995313 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:38.996460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:38.996460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:38.996460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:38.996460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:38.996460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:38.996460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:41:38.996460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:38.996460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:38.996460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:38.996460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:38.996460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:38.996460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:38.997200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:38.997200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:38.997200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:38.997200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:38.997200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:38.997200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:38.997200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:38.997200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:38.997200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:38.997200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:38.997200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:38.997200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:38.997200 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:38.998134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:38.998134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:38.998134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:38.998134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:38.998134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:38.998134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:38.998134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:38.998134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:38.998134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:38.998134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:38.998134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:38.998134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:38.998134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:38.998134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:38.998134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:38.999984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:38.999984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:38.999984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:38.999984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:41:38.999984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:38.999984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:38.999984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:38.999984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:38.999984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:38.999984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:38.999984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:38.999984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:38.999984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:38.999984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:38.999984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:38.999984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:38.999984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:38.999984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:38.999984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:39.000899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:39.000899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:39.000899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:39.000899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:39.000899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:39.000899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:39.019194 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:39.019475 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:39.019571 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:39.019670 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:41:39.020479 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:39.020942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:39.021841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:39.021841 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:39.021841 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:39.021841 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:39.021841 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:39.021841 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:39.021841 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:39.021841 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:39.022612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:39.022612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:41:39.022612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:39.022612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:39.022612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:39.022612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:39.022612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:39.022612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:39.022612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:39.022612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:39.022612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:39.022612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:41:39.022612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:39.022612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:41:39.022612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:41:39.023116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:39.023116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:41:39.023116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:39.023116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:41:39.023116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:41:39.023116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:39.023116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:41:39.023116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:39.023116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:41:39.023401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:39.023401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:39.023401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:39.023401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:39.023401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:39.023401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:41:39.023401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:41:39.023401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:41:39.023401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:39.023401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:39.023401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:39.023401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:39.023763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:39.023763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:39.023763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:39.023763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:39.023763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:39.023763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:39.023763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:39.023763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:39.023763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:39.023763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:39.023763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:39.023763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:39.023763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:39.023763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:39.024208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:39.024208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:39.024208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:39.024208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:39.024481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:39.024481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:39.024481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:39.024481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:39.024481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:39.024481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:39.024481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:39.024481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:39.024481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:39.024481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:39.024481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:39.024481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:39.024481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:39.024481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:39.024481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:39.024481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:39.025424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:41:39.025424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:41:39.025424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:41:39.025424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:39.025424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:39.025424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:39.025424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:39.025424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:39.025424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:39.025424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:39.025424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:39.025424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:39.025424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:39.025424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:39.025424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:39.025424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:39.025424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:39.025424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:39.026421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:39.026421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:39.026421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:39.026421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:39.026421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:39.026421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:39.026421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:39.040569 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:39.040888 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:39.040969 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:39.041090 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:39.041760 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:39.042270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:39.043162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:39.043162 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:39.043162 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:39.043162 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:39.043162 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:39.043162 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:39.043162 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:39.043162 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:39.043905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:39.043905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:41:39.043905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:39.043905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:39.043905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:39.043905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:39.043905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:39.043905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:39.043905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:39.043905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:39.043905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:39.043905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:41:39.043905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:39.043905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:41:39.043905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:41:39.044368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:39.044368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:41:39.044368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:39.044368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:41:39.044368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:41:39.044368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:39.044368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:41:39.044368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:39.044368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:41:39.044801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:39.044801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:39.044801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:39.044801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:39.044801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:39.044801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:41:39.044801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:41:39.044801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:39.044801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:39.044801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:39.044801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:39.045275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:39.045275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:39.045275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:39.045275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:39.045275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:39.045275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:39.045275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:39.045275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:39.045275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:39.045275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:39.045275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:39.045275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:39.045275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:39.045275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:39.045722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:39.045722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:39.045722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:39.045722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:39.045722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:39.045722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:39.045722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:39.045722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:39.045722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:39.045722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:39.045722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:39.045722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:39.045722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:39.045722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:39.046110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:39.046110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:39.046110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:39.046110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:39.046110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:39.046110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:39.046110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:41:39.046110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:41:39.046110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:39.046110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:39.046110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:39.046110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:39.046110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:39.046110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:39.046110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:39.046110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:39.046110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:39.046110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:39.046110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:39.046651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:39.046651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:39.046651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:39.046651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:39.046651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:39.046651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:39.046651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:39.046651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:39.046651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:39.046651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:39.046651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:39.112816 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:39.113076 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:39.113396 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:39.113396 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:39.114294 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:39.114841 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:39.115119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:39.116133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:39.116133 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:39.116133 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:39.116133 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:39.116133 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:39.116133 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:39.116133 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:39.116133 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:39.116992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:39.116992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:39.116992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:39.116992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:39.116992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:39.116992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:41:39.116992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:39.116992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:39.116992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:39.116992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:39.116992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:39.116992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:39.117598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:39.117598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:39.117598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:39.117598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:39.117598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:39.117598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:39.117598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:39.117598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:39.117598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:39.117598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:39.117598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:39.117598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:39.117598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:39.118238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:39.118238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:39.118238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:39.118238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:39.118238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:39.118238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:39.118238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:39.118238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:39.118238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:39.118238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:39.118238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:39.118238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:39.118238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:39.118238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:39.118238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:39.118954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:39.118954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:39.118954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:39.118954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:41:39.118954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:39.118954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:39.118954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:39.118954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:39.118954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:39.118954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:39.118954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:39.118954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:39.118954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:39.118954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:39.118954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:39.118954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:39.118954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:39.118954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:39.118954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:39.119856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:39.119856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:39.119856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:39.119856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:39.119856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:39.119856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:43.778449 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:43.779250 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:43.779250 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:43.780463 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:43.815531 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:43.816224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:43.817169 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:43.817169 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:43.817169 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:43.817169 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:43.817169 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:43.817169 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:43.817169 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:43.817169 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:43.818187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:43.818187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:43.818187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:43.818187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:43.818187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:43.818187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:41:43.818187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:43.818187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:43.818187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:43.818187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:43.818187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:43.818187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:43.819320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:43.819320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:43.819320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:43.819320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:43.819320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:43.819320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:43.819320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:43.819320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:43.819320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:43.819320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:43.819320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:43.819320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:43.819320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:43.819892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:43.819892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:43.819892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:43.819892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:43.819892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:43.819892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:43.819892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:43.819892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:43.819892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:43.819892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:43.819892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:43.819892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:43.819892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:43.819892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:43.819892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:43.820574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:43.820574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:43.820574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:43.820574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:41:43.820574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:43.820574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:43.820574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:43.820574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:43.820574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:43.820574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:43.820574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:43.820574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:43.820574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:43.820574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:43.820574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:43.820574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:43.820574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:43.820574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:43.820574 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:43.821658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:43.821658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:43.821658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:43.821658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:43.821658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:43.821658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:43.992446 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:43.992892 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:43.993062 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:43.993640 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:43.993869 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:43.993990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:43.994762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:43.994762 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:43.994762 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:43.994762 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:43.994762 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:43.994762 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:43.994762 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:43.994762 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:43.995642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:43.995642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:43.995642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:43.995642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:43.995642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:43.995642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:41:43.995642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:43.995642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:43.995642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:43.995642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:43.995642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:43.995642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:43.996833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:43.996833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:43.996833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:43.996833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:43.996833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:43.996833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:43.996833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:43.996833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:43.996833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:43.996833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:43.996833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:43.996833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:43.996833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:43.998205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:43.998205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:43.998205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:43.998205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:43.998205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:43.998205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:43.998205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:43.998205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:43.998205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:43.998205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:43.998205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:43.998205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:43.998205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:43.998205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:43.998205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:43.999712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:43.999712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:43.999712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:43.999712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:41:43.999712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:43.999712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:43.999712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:43.999712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:43.999712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:43.999712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:43.999712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:43.999712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:43.999712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:43.999712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:43.999712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:43.999712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:43.999712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:43.999712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:43.999712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:44.000708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:44.000708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:44.000708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:44.000708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:44.000708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:44.000708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:44.019655 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:44.020175 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:44.020343 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:44.020569 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:41:44.020972 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:44.021144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:44.021795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:44.021795 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:44.021795 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:44.021795 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:44.021795 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:44.021795 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:44.021795 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:44.021795 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:44.022535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:44.022535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:41:44.022535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:44.022535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:44.022535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:44.022535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:44.022535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:44.022535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:44.022535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:44.022535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:44.022535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:44.022535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:41:44.022535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:44.022535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:41:44.022535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:41:44.023400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:44.023400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:41:44.023400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:44.023400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:41:44.023400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:41:44.023400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:44.023400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:41:44.023400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:44.023400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:41:44.023400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:44.023400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:44.023400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:44.023400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:44.023400 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:44.024096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:41:44.024096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:41:44.024096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:41:44.024096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:44.024096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:44.024096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:44.024096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:44.024096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:44.024096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:44.024096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:44.024096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:44.024096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:44.024096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:44.024615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:44.024615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:44.024615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:44.024615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:44.024615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:44.024615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:44.024615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:44.024615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:44.024615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:44.024615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:44.024615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:44.024615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:44.026047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:44.026047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:44.026047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:44.026047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:44.026047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:44.026047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:44.026047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:44.026047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:44.026047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:44.026047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:44.026047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:44.026047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:44.026047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:44.026047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:44.026047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:44.026047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:44.026455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:41:44.026455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:41:44.026455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:41:44.026455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:44.026455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:44.026455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:44.026455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:44.026455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:44.026455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:44.026455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:44.026455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:44.026455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:44.026455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:44.026455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:44.026455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:44.026455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:44.026455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:44.026455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:44.026911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:44.026911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:44.026911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:44.026911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:44.026911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:44.026911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:44.026911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:44.041145 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:44.041545 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:44.041850 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:44.042275 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:44.042644 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:44.042890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:44.043428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:44.043428 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:44.043428 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:44.043428 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:44.043428 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:44.043428 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:44.043428 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:44.043428 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:44.045205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:44.045205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:41:44.045205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:44.045205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:44.045205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:44.045205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:44.045205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:44.045205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:44.045205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:44.045205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:44.045205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:44.045205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:41:44.045205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:44.045205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:41:44.045205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:41:44.046081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:44.046081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:41:44.046081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:44.046081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:41:44.046081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:41:44.046081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:44.046081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:41:44.046081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:44.046081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:41:44.046471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:44.046471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:44.046471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:44.046471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:44.046471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:44.046471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:41:44.046471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:41:44.046471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:44.046471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:44.046471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:44.046471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:44.046965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:44.046965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:44.046965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:44.046965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:44.046965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:44.046965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:44.046965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:44.046965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:44.046965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:44.046965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:44.046965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:44.046965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:44.046965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:44.046965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:44.047551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:44.047551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:44.047551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:44.047551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:44.047551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:44.047551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:44.047551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:44.047551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:44.047551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:44.047551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:44.047551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:44.047551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:44.047551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:44.047551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:44.048180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:44.048180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:44.048180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:44.048180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:44.048180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:44.048180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:44.048180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:41:44.048180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:41:44.048180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:44.048180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:44.048180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:44.048180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:44.048180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:44.048180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:44.048180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:44.048180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:44.048180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:44.048180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:44.048180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:44.049121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:44.049121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:44.049121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:44.049121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:44.049121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:44.049121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:44.049121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:44.049121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:44.049121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:44.049121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:44.049121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:44.113354 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:44.113838 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:44.114099 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:44.114382 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:44.115473 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:44.116024 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:44.116290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:44.117422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:44.117422 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:44.117422 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:44.117422 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:44.117422 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:44.117422 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:44.117422 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:44.117422 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:44.118878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:44.118878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:44.118878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:44.118878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:44.118878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:44.118878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:41:44.118878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:44.118878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:44.118878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:44.118878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:44.118878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:44.118878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:44.119532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:44.119532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:44.119532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:44.119532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:44.119532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:44.119532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:44.119532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:44.119532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:44.119532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:44.119532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:44.119532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:44.119532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:44.119532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:44.120132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:44.120132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:44.120478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:44.120478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:44.120478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:44.120478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:44.120478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:44.120478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:44.120478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:44.120478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:44.120478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:44.120478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:44.120478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:44.120478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:44.120478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:44.120478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:44.120478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:44.120478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:44.121366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:41:44.121366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:44.121366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:44.121366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:44.121366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:44.121366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:44.121366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:44.121366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:44.121366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:44.121366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:44.121366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:44.121366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:44.121366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:44.121366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:44.121366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:44.121366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:44.121366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:44.121366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:44.121366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:44.122222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:44.122222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:44.122222 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:48.779164 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:48.779656 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:48.779815 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:48.780637 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:48.811564 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:48.812048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:48.812854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:48.812854 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:48.812854 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:48.812854 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:48.812854 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:48.812854 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:48.812854 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:48.812854 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:48.813840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:48.813840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:48.813840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:48.813840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:48.813840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:48.813840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:41:48.813840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:48.813840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:48.813840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:48.813840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:48.813840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:48.813840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:48.814196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:48.814196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:48.814196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:48.814196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:48.814196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:48.814196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:48.814196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:48.814196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:48.814196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:48.814196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:48.814196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:48.814196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:48.814196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:48.814562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:48.814562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:48.815198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:48.815198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:48.815198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:48.815198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:48.815198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:48.815198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:48.815198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:48.815198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:48.815198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:48.815198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:48.815198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:48.815198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:48.815198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:48.815198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:48.815198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:48.815198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:48.816332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:41:48.816332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:48.816332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:48.816332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:48.816332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:48.816332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:48.816332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:48.816332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:48.816332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:48.816332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:48.816332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:48.816332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:48.816332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:48.816332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:48.816332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:48.816332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:48.816332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:48.816332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:48.816332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:48.817479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:48.817479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:48.817479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:48.993180 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:48.994145 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:48.994145 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:48.995043 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:48.995398 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:48.996111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:48.996646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:48.996646 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:48.996646 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:48.996646 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:48.996646 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:48.996646 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:48.996646 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:48.996646 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:48.998328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:48.998328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:48.998328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:48.998328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:48.998328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:48.998328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:41:48.998328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:48.998328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:48.998328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:48.998328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:48.998328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:48.998328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:48.999041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:48.999041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:48.999041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:48.999041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:48.999041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:48.999041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:48.999041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:48.999041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:48.999041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:48.999041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:48.999041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:48.999041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:48.999041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:48.999790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:48.999790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:49.000039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:49.000039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:49.000039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:49.000039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:49.000039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:49.000039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:49.000039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:49.000039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:49.000039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:49.000039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:49.000039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:49.000039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:49.000039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:49.000039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:49.000039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:49.000039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:49.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:41:49.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:49.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:49.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:49.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:49.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:49.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:49.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:49.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:49.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:49.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:49.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:49.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:49.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:49.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:49.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:49.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:49.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:49.001620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:49.003303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:49.003303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:49.003303 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:49.020265 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:49.020825 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:49.021026 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:49.021233 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:41:49.021913 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:49.022260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:49.023328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:49.023328 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:49.023328 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:49.023328 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:49.023328 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:49.023328 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:49.023328 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:49.023328 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:49.024525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:49.024525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:41:49.024525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:49.024525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:49.024525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:49.024525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:49.024525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:49.024525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:49.024525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:49.024525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:49.024525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:49.024525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:41:49.024525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:49.024525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:41:49.024525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:41:49.025348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:49.025348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:41:49.025348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:49.025348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:41:49.025348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:41:49.025348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:49.025348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:41:49.025348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:49.025348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:41:49.026191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:49.026191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:49.026191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:49.026191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:49.026191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:49.026191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:41:49.026191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:41:49.026191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:41:49.026191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:49.026191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:49.026191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:49.026191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:49.027274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:49.027274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:49.027274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:49.027274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:49.027274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:49.027274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:49.027274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:49.027274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:49.027274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:49.027274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:49.027274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:49.027274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:49.027274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:49.027274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:49.028549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:49.028549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:49.028549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:49.028549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:49.028549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:49.028549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:49.028549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:49.028549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:49.028549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:49.028549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:49.028549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:49.028549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:49.028549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:49.028549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:49.030202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:49.030202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:49.030202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:49.030202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:49.030202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:49.030202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:49.030202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:41:49.030202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:41:49.030202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:41:49.030202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:49.030202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:49.030202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:49.030202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:49.030202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:49.030202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:49.030202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:49.030202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:49.030202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:49.030202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:49.030694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:49.030694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:49.030694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:49.030694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:49.030694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:49.030694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:49.030694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:49.030694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:49.030694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:49.030694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:49.030694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:49.030694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:49.041492 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:49.041777 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:49.041879 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:49.042011 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:49.042373 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:49.042689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:49.043577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:49.043577 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:49.043577 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:49.043577 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:49.043577 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:49.043577 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:49.043577 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:49.043577 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:49.045238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:49.045238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:41:49.045238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:49.045238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:49.045238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:49.045238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:49.045238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:49.045238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:49.045238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:49.045238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:49.045238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:49.045238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:41:49.045238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:49.045238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:41:49.045238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:41:49.045690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:49.045690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:41:49.045690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:49.045690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:41:49.045690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:41:49.045690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:49.045690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:41:49.045690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:49.045690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:41:49.046667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:49.046667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:49.046667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:49.046667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:49.046667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:49.046667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:41:49.046667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:41:49.046667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:49.046667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:49.046667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:49.046667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:49.047206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:49.047206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:49.047206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:49.047206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:49.047206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:49.047206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:49.047206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:49.047206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:49.047206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:49.047206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:49.047206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:49.047206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:49.047206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:49.047206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:49.047572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:49.047572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:49.047572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:49.047572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:49.048482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:49.048482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:49.048482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:49.048482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:49.048482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:49.048482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:49.048482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:49.048482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:49.048482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:49.048482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:49.048482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:49.048482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:49.048482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:49.048482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:49.048482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:49.048482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:49.048482 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:41:49.049012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:41:49.049012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:49.049012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:49.049012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:49.049012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:49.049012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:49.049012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:49.049012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:49.049012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:49.049012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:49.049012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:49.049012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:49.049012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:49.049012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:49.049012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:49.049012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:49.049012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:49.049012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:49.049012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:49.049515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:49.049515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:49.049515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:49.049515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:49.113971 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:49.114366 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:49.114723 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:49.114816 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:49.115375 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:49.115877 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:49.116172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:49.116956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:49.116956 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:49.116956 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:49.116956 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:49.116956 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:49.116956 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:49.116956 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:49.116956 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:49.118247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:49.118247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:49.118247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:49.118247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:49.118247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:49.118247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:41:49.118247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:49.118247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:49.118247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:49.118247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:49.118247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:49.118247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:49.118949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:49.118949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:49.118949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:49.118949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:49.118949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:49.118949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:49.118949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:49.118949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:49.118949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:49.118949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:49.118949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:49.118949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:49.118949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:49.119506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:49.119506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:49.119963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:49.119963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:49.119963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:49.119963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:49.119963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:49.119963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:49.119963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:49.119963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:49.119963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:49.119963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:49.119963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:49.119963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:49.119963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:49.119963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:49.119963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:49.119963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:49.120739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:41:49.120739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:49.120739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:49.120739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:49.120739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:49.120739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:49.120739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:49.120739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:49.120739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:49.120739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:49.120739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:49.120739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:49.120739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:49.120739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:49.120739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:49.120739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:49.120739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:49.120739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:49.120739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:49.121542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:49.121542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:49.121542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:53.779849 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:53.780478 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:53.780711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:53.782054 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:53.818959 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:53.819411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:53.820445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:53.820445 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:53.820445 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:53.820445 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:53.820445 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:53.820445 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:53.820445 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:53.820445 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:53.821259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:53.821259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:53.821259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:53.821259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:53.821259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:53.821259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:41:53.821259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:53.821259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:53.821259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:53.821259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:53.821259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:53.821259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:53.821752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:53.821752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:53.821752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:53.821752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:53.821752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:53.821752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:53.821752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:53.821752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:53.821752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:53.821752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:53.821752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:53.821752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:53.821752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:53.822163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:53.822163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:53.822163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:53.822163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:53.822163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:53.822163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:53.822163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:53.822163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:53.822163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:53.822163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:53.822163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:53.822163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:53.822163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:53.822163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:53.822163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:53.822723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:53.822723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:53.822723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:53.822723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:41:53.822723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:53.822723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:53.822723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:53.822723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:53.822723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:53.822723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:53.822723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:53.822723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:53.822723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:53.822723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:53.822723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:53.822723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:53.822723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:53.822723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:53.822723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:53.823405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:53.823405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:53.823405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:53.823405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:53.823405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:53.823405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:53.993705 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:53.994290 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:53.994523 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:53.995422 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:53.995613 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:53.995814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:53.996429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:53.996429 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:53.996429 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:53.996429 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:53.996429 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:53.996429 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:53.996429 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:53.996429 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:53.998209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:53.998209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:53.998209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:53.998209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:53.998209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:53.998209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:41:53.998209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:53.998209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:53.998209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:53.998209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:53.998209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:53.998209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:53.998776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:53.998776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:53.998776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:53.998776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:53.998776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:53.998776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:53.998776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:53.998776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:53.998776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:53.998776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:53.998776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:53.998776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:53.998776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:53.999337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:53.999337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:54.000270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:54.000270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:54.000270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:54.000270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:54.000270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:54.000270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:54.000270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:54.000270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:54.000270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:54.000270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:54.000270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:54.000270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:54.000270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:54.000270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:54.000270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:54.000270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:54.001253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:41:54.001253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:54.001253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:54.001253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:54.001253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:54.001253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:54.001253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:54.001253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:54.001253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:54.001253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:54.001253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:54.001253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:54.001253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:54.001253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:54.001253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:54.001253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:54.001253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:54.001253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:54.001253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:54.002041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:54.002041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:54.002041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:54.020834 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:54.021168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:54.021295 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:54.021423 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:41:54.021846 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:54.022168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:54.022932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:54.022932 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:54.022932 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:54.022932 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:54.022932 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:54.022932 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:54.022932 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:54.022932 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:54.023894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:54.023894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:41:54.023894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:54.023894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:54.023894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:54.023894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:54.023894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:54.023894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:54.023894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:54.023894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:54.023894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:54.023894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:41:54.023894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:54.023894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:41:54.023894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:41:54.024286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:54.024286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:41:54.024286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:54.024286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:41:54.024286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:41:54.024286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:54.024286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:41:54.024286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:54.024286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:41:54.024797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:54.024797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:54.024797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:54.024797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:54.024797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:54.024797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:41:54.024797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:41:54.024797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:41:54.024797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:54.024797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:54.024797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:54.024797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:54.025180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:54.025180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:54.025180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:54.025180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:54.025180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:54.025180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:54.025180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:54.025180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:54.025180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:54.025180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:54.025180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:54.025180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:54.025180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:54.025180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:54.025530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:54.025530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:54.025530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:54.025530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:54.025827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:54.025827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:54.025827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:54.025827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:54.025827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:54.025827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:54.025827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:54.025827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:54.025827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:54.025827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:54.025827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:54.025827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:54.025827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:54.025827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:54.025827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:54.025827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:54.026545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:41:54.026545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:41:54.026545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:41:54.026545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:54.026545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:54.026545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:54.026545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:54.026545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:54.026545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:54.026545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:54.026545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:54.026545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:54.026545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:54.026545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:54.026545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:54.026545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:54.026545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:54.026545 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:54.027387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:54.027387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:54.027387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:54.027387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:54.027387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:54.027387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:54.027387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:54.042000 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:54.042369 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:54.042572 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:54.042824 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:54.043343 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:54.043759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:54.044561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:54.044561 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:54.044561 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:54.044561 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:54.044561 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:54.044561 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:54.044561 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:54.044561 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:54.045886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:54.045886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:41:54.045886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:54.045886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:54.045886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:54.045886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:54.045886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:54.045886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:54.045886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:54.045886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:54.045886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:54.045886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:41:54.045886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:54.045886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:41:54.045886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:41:54.046568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:54.046568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:41:54.046568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:54.046568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:41:54.046568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:41:54.046568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:54.046568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:41:54.046568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:54.046568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:41:54.047468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:54.047468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:54.047468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:54.047468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:54.047468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:54.047468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:41:54.047468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:41:54.047468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:54.047468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:54.047468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:54.047468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:54.048026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:54.048026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:54.048026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:54.048026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:54.048026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:54.048026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:54.048026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:54.048026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:54.048026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:54.048026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:54.048026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:54.048026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:54.048026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:54.048026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:54.048711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:54.048711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:54.048711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:54.048711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:54.049395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:54.049395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:54.049395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:54.049395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:54.049395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:54.049395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:54.049395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:54.049395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:54.049395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:54.049395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:54.049395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:54.049395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:54.049395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:54.049395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:54.049395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:54.049395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:54.049395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:41:54.050146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:41:54.050146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:54.050146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:54.050146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:54.050146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:54.050146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:54.050146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:54.050146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:54.050146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:54.050146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:54.050146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:54.050146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:54.050146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:54.050146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:54.050146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:54.050146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:54.050146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:54.050146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:54.050146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:54.050986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:54.050986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:54.050986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:54.050986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:54.114489 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:54.115056 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:54.115309 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:54.115623 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:54.116454 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:54.116991 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:54.117233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:54.118125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:54.118125 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:54.118125 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:54.118125 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:54.118125 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:54.118125 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:54.118125 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:54.118125 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:54.119871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:54.119871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:54.119871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:54.119871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:54.119871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:54.119871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:41:54.119871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:54.119871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:54.119871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:54.119871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:54.119871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:54.119871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:54.120234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:54.120234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:54.120234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:54.120234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:54.120234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:54.120234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:54.120234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:54.120234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:54.120234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:54.120234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:54.120234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:54.120234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:54.120234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:54.120538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:54.120538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:54.121982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:54.121982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:54.121982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:54.121982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:54.121982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:54.121982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:54.121982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:54.121982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:54.121982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:54.121982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:54.121982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:54.121982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:54.121982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:54.121982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:54.121982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:54.121982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:54.122359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:41:54.122359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:54.122359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:54.122359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:54.122359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:54.122359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:54.122359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:54.122359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:54.122359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:54.122359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:54.122359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:54.122359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:54.122359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:54.122359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:54.122359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:54.122359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:54.122359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:54.122359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:54.122359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:54.122816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:54.122816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:54.122816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:58.780563 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:58.781309 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:58.781309 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:58.782474 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:58.817653 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:58.818145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:58.819025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:58.819025 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:58.819025 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:58.819025 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:58.819025 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:58.819025 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:58.819025 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:58.819025 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:58.820148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:58.820148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:58.820148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:58.820148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:58.820148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:58.820148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:41:58.820148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:58.820148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:58.820148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:58.820148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:58.820148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:58.820148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:58.820770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:58.820770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:58.820770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:58.820770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:58.820770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:58.820770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:58.820770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:58.820770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:58.820770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:58.820770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:58.820770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:58.820770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:58.820770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:58.821601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:58.821601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:58.821601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:58.821601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:58.821601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:58.821601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:58.821601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:58.821601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:58.821601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:58.821601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:58.821601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:58.821601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:58.821601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:58.821601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:58.821601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:58.822628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:58.822628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:58.822628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:58.822628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:41:58.822628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:58.822628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:58.822628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:58.822628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:58.822628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:58.822628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:58.822628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:58.822628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:58.822628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:58.822628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:58.822628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:58.822628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:58.822628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:58.822628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:58.822628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:58.824018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:58.824018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:58.824018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:58.824018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:58.824018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:58.824018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:58.994584 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:58.995057 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:58.995167 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:58.996390 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:58.996765 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:58.997013 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:58.997818 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:58.997818 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:58.997818 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:58.997818 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:58.997818 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:58.997818 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:58.997818 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:58.997818 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:58.998868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:58.998868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:58.998868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:58.998868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:58.998868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:58.998868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:41:58.998868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:58.998868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:58.998868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:58.998868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:58.998868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:58.998868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:58.999451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:58.999451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:58.999451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:58.999451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:58.999451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:58.999451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:58.999451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:58.999451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:58.999451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:58.999451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:58.999451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:58.999451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:58.999451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:59.000120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.000120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:59.000120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.000120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:59.000120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:59.000120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:59.000120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:59.000120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:59.000120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:59.000120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:59.000120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:59.000120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:59.000120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:59.000120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:59.000120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:59.000871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:59.000871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:59.000871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:59.000871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:41:59.000871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:59.000871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:59.000871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:59.000871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:59.000871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:59.000871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:59.000871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:59.000871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:59.000871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:59.000871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:59.000871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:59.000871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:59.000871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:59.000871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:59.000871 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:59.001782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:59.001782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:59.001782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:59.001782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:59.001782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.001782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:59.021269 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:59.021505 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:59.021619 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:59.021822 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:41:59.022359 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:59.022603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.023323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:59.023323 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:59.023323 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:59.023323 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:59.023323 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:59.023323 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:59.023323 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.023323 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:59.024126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.024126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:41:59.024126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:59.024126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:59.024126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:59.024126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:59.024126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:59.024126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:59.024126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:59.024126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:59.024126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:59.024126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:41:59.024126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:59.024126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:41:59.024126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:41:59.024829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:59.024829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:41:59.024829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:59.024829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:41:59.024829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:41:59.024829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:59.024829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:41:59.024829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.024829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:41:59.025263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.025263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:59.025263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:59.025263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:59.025263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:59.025263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:41:59.025263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:41:59.025263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:41:59.025263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:59.025263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:59.025263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:59.025263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:59.025900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:59.025900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:59.025900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:59.025900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:59.025900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:59.025900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:59.025900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:59.025900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:59.025900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:59.025900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:59.025900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:59.025900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:59.025900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:59.025900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:59.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:59.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:59.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:59.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:59.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:59.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:59.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:59.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:59.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:59.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:59.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:59.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:59.027269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:59.027269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:59.027269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:59.027269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:59.027269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:59.027269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:59.027269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:41:59.027269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:41:59.027269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:41:59.027269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:59.027269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:59.027269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:59.027269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:59.027269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:59.027269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:59.027269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:59.027269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:59.027269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:59.027269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:59.028173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:59.028173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:59.028173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:59.028173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:59.028173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:59.028173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:59.028173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:59.028173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:59.028173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:59.028173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:59.028173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.028173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:59.042391 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:59.042771 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:59.042943 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:59.043136 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:59.043691 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:59.044081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.044802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:59.044802 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:59.044802 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:59.044802 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:59.044802 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:59.044802 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:59.044802 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.044802 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:59.046007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.046007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:41:59.046007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:59.046007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:59.046007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:59.046007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:59.046007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:59.046007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:59.046007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:59.046007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:41:59.046007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:59.046007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:41:59.046007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:59.046007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:41:59.046007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:41:59.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:59.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:41:59.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:59.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:41:59.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:41:59.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:59.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:41:59.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.046645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:41:59.047397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.047397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:59.047397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:59.047397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:59.047397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:59.047397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:41:59.047397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:41:59.047397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:59.047397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:59.047397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:59.047397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:59.048080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:59.048080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:59.048080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:59.048080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:59.048080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:59.048080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:59.048080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:59.048080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:59.048080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:59.048080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:59.048080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:59.048080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:59.048080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:59.048080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:59.048744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:59.048744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:59.048744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.048744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:59.049262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.049262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:59.049262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:59.049262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:59.049262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:59.049262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:59.049262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:59.049262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:59.049262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:59.049262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:59.049262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:59.049262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:59.049262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:59.049262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:59.049262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:59.049262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:59.049262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:41:59.050112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:41:59.050112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:41:59.050112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:41:59.050112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:59.050112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:59.050112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:59.050112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:59.050112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:59.050112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:59.050112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:59.050112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:59.050112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:59.050112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:59.050112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:59.050112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:59.050112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:59.050112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:59.050112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:59.050112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:59.051045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:59.051045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:59.051045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.051045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:41:59.115036 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:41:59.115578 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:41:59.115806 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:41:59.116087 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:41:59.116851 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:59.117273 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:41:59.118362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.118587 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:41:59.118587 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:59.118587 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:41:59.118587 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:41:59.118587 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:59.118587 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:41:59.118587 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.118587 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:41:59.119748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.119748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:41:59.119748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:59.119748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:41:59.119748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:59.119748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:41:59.119748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:59.119748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:59.119748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:59.119748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:59.119748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:59.119748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:59.120229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:41:59.120229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:59.120229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:41:59.120229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:59.120229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:41:59.120229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:41:59.120229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:59.120229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:41:59.120229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:59.120229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:41:59.120229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:41:59.120229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:59.120229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:41:59.120634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.120634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:41:59.121360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.121360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:41:59.121360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:41:59.121360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:41:59.121360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:41:59.121360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:41:59.121360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:59.121360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:41:59.121360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:41:59.121360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:59.121360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:41:59.121360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:41:59.121360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:41:59.121360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:41:59.121360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:41:59.121360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:41:59.122196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:41:59.122196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:41:59.122196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:59.122196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:59.122196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:41:59.122196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:41:59.122196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:41:59.122196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:41:59.122196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:41:59.122196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:41:59.122196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:41:59.122196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:41:59.122196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:41:59.122196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:41:59.122196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:41:59.122196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:41:59.122196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:41:59.122196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:41:59.122196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:41:59.122859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:41:59.122859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:41:59.122859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:03.781188 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:03.781931 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:03.781931 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:03.782587 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:03.813649 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:03.814230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:03.815478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:03.815478 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:03.815478 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:03.815478 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:03.815478 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:03.815478 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:03.815478 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:03.815478 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:03.817133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:03.817133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:03.817133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:03.817133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:03.817133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:03.817133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:42:03.817133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:03.817133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:03.817133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:03.817133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:03.817133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:03.817133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:03.817789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:03.817789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:03.817789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:03.817789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:03.817789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:03.817789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:03.817789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:03.817789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:03.817789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:03.817789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:03.817789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:03.817789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:03.817789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:03.818349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:03.818349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:03.818903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:03.818903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:03.818903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:03.818903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:03.818903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:03.818903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:03.818903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:03.818903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:03.818903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:03.818903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:03.818903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:03.818903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:03.818903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:03.818903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:03.818903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:03.818903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:03.819641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:42:03.819641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:03.819641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:03.819641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:03.819641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:03.819641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:03.819641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:03.819641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:03.819641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:03.819641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:03.819641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:03.819641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:03.819641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:03.819641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:03.819641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:03.819641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:03.819641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:03.819641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:03.819641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:03.820505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:03.820505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:03.820505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:03.995190 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:03.995718 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:03.995911 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:03.997260 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:03.997619 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:03.997903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:03.998927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:03.998927 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:03.998927 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:03.998927 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:03.998927 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:03.998927 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:03.998927 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:03.998927 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:04.000717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.000717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:04.000717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:04.000717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:04.000717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:04.000717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:42:04.000717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:04.000717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:04.000717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:04.000717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:04.000717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:04.000717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:04.001311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:04.001311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:04.001311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:04.001311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:04.001311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:04.001311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:04.001311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:04.001311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:04.001311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:04.001311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:04.001311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:04.001311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:04.001311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:04.001861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.001861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:04.002583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.002583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:04.002583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:04.002583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:04.002583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:04.002583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:04.002583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:04.002583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:04.002583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:04.002583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:04.002583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:04.002583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:04.002583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:04.002583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:04.002583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:04.002583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:04.003269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:42:04.003269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:04.003269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:04.003269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:04.003269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:04.003269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:04.003269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:04.003269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:04.003269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:04.003269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:04.003269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:04.003269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:04.003269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:04.003269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:04.003269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:04.003269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:04.003269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:04.003269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:04.003269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:04.003865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:04.003865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.003865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:04.021697 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:04.022059 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:04.022164 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:04.022277 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:42:04.022878 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:04.023230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.024077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:04.024077 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:04.024077 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:04.024077 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:04.024077 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:04.024077 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:04.024077 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.024077 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:04.024944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.024944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:42:04.024944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:04.024944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:04.024944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:04.024944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:04.024944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:04.024944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:04.024944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:04.024944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:04.024944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:04.024944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:42:04.024944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:04.024944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:42:04.024944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:42:04.025720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:04.025720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:42:04.025720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:04.025720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:42:04.025720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:42:04.025720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:04.025720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:42:04.025720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.025720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:42:04.026166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.026166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:04.026166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:04.026166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:04.026166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:04.026166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:42:04.026166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:42:04.026166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:42:04.026166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:04.026166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:04.026166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:04.026166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:04.026753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:04.026753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:04.026753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:04.026753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:04.026753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:04.026753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:04.026753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:04.026753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:04.026753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:04.026753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:04.026753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:04.026753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:04.026753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:04.026753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:04.027346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:04.027346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:04.027346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.027346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:04.027346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.027346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:04.027346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:04.027346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:04.027346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:04.027346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:04.027346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:04.027346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:04.027346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:04.027346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:04.027965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:04.027965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:04.027965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:04.027965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:04.027965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:04.027965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:04.027965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:42:04.027965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:42:04.027965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:42:04.027965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:04.027965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:04.027965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:04.027965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:04.027965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:04.027965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:04.027965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:04.027965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:04.027965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:04.027965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:04.028906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:04.028906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:04.028906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:04.028906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:04.028906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:04.028906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:04.028906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:04.028906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:04.028906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:04.028906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:04.028906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.028906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:04.042902 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:04.043309 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:04.043496 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:04.043732 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:04.044331 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:04.044690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.045445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:04.045445 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:04.045445 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:04.045445 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:04.045445 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:04.045445 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:04.045445 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.045445 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:04.046559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.046559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:42:04.046559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:04.046559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:04.046559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:04.046559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:04.046559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:04.046559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:04.046559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:04.046559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:04.046559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:04.046559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:42:04.046559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:04.046559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:42:04.046559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:42:04.047379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:04.047379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:42:04.047379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:04.047379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:42:04.047379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:42:04.047379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:04.047379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:42:04.047379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.047379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:42:04.047379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.047379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:04.047379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:04.047379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:04.047379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:04.047951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:42:04.047951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:42:04.047951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:04.047951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:04.047951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:04.047951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:04.047951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:04.047951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:04.047951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:04.047951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:04.047951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:04.047951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:04.047951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:04.048431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:04.048431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:04.048431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:04.048431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:04.048431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:04.048431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:04.048431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:04.048431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:04.048431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:04.048431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.048431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:04.048431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.048910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:04.048910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:04.048910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:04.048910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:04.048910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:04.048910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:04.048910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:04.048910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:04.048910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:04.048910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:04.048910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:04.048910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:04.048910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:04.048910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:04.048910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:04.048910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:42:04.048910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:42:04.048910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:04.048910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:04.049630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:04.049630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:04.049630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:04.049630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:04.049630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:04.049630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:04.049630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:04.049630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:04.049630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:04.049630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:04.049630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:04.049630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:04.049630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:04.049630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:04.049630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:04.049630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:04.049630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:04.049630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:04.049630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.049630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:04.115600 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:04.116314 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:04.116418 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:04.116652 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:04.117210 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:04.117769 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:04.118095 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.119027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:04.119027 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:04.119027 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:04.119027 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:04.119027 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:04.119027 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:04.119027 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.119027 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:04.120417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.120417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:04.120417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:04.120417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:04.120417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:04.120417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:42:04.120417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:04.120417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:04.120417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:04.120417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:04.120417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:04.120417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:04.121018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:04.121018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:04.121018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:04.121018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:04.121018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:04.121018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:04.121018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:04.121018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:04.121018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:04.121018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:04.121018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:04.121018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:04.121018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:04.121573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.121573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:04.121976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.121976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:04.121976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:04.121976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:04.121976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:04.121976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:04.121976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:04.121976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:04.121976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:04.121976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:04.121976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:04.121976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:04.121976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:04.121976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:04.121976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:04.121976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:04.122779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:42:04.122779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:04.122779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:04.122779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:04.122779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:04.122779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:04.122779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:04.122779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:04.122779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:04.122779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:04.122779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:04.122779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:04.122779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:04.122779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:04.122779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:04.122779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:04.122779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:04.122779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:04.122779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:04.123701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:04.123701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:04.123701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:08.781898 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:08.782630 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:08.782630 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:08.783943 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:08.786702 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=434 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:42:08.818868 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:08.819259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:08.820267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:08.820267 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:08.820267 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:08.820267 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:08.820267 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:08.820267 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:08.820267 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:08.820267 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:08.821973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:08.821973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:08.821973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:08.821973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:08.821973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:08.821973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:42:08.821973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:08.821973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:08.821973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:08.821973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:08.821973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:08.821973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:08.822595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:08.822595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:08.822595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:08.822595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:08.822595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:08.822595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:08.822595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:08.822595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:08.822595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:08.822595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:08.822595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:08.822595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:08.822595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:08.823161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:08.823161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:08.823613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:08.823613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:08.823613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:08.823613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:08.823613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:08.823613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:08.823613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:08.823613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:08.823613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:08.823613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:08.823613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:08.823613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:08.823613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:08.823613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:08.823613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:08.823613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:08.824335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:42:08.824335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:08.824335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:08.824335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:08.824335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:08.824335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:08.824335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:08.824335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:08.824335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:08.824335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:08.824335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:08.824335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:08.824335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:08.824335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:08.824335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:08.824335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:08.824335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:08.824335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:08.824335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:08.825110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:08.825110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:08.825110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:08.995788 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:08.996105 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:08.996222 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:08.997559 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:08.997882 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:08.998116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:08.999038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:08.999038 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:08.999038 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:08.999038 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:08.999038 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:08.999038 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:08.999038 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:08.999038 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:09.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:09.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:09.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:09.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:09.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:42:09.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:09.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:09.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:09.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:09.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:09.000843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:09.001468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:09.001468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:09.001468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:09.001468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:09.001468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:09.001468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:09.001468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:09.001468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:09.001468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:09.001468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:09.001468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:09.001468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:09.001468 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:09.002091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.002091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:09.002553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.002553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:09.002553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:09.002553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:09.002553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:09.002553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:09.002553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:09.002553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:09.002553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:09.002553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:09.002553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:09.002553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:09.002553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:09.002553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:09.002553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:09.002553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:09.003420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:42:09.003420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:09.003420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:09.003420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:09.003420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:09.003420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:09.003420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:09.003420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:09.003420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:09.003420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:09.003420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:09.003420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:09.003420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:09.003420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:09.003420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:09.003420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:09.003420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:09.003420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:09.003420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:09.004248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:09.004248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.004248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:09.022228 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:09.022821 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:09.023023 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:09.023219 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:42:09.023747 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:09.023958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.024689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:09.024689 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:09.024689 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:09.024689 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:09.024689 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:09.024689 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:09.024689 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.024689 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:09.026177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.026177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:42:09.026177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:09.026177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:09.026177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:09.026177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:09.026177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:09.026177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:09.026177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:09.026177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:09.026177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:09.026177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:42:09.026177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:09.026177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:42:09.026177 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:42:09.027137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:09.027137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:42:09.027137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:09.027137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:42:09.027137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:42:09.027137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:09.027137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:42:09.027137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.027137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:42:09.027575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.027575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:09.027575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:09.027575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:09.027575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:09.027575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:42:09.027575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:42:09.027575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:42:09.027575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:09.027575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:09.027575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:09.027575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:09.028176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:09.028176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:09.028176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:09.028176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:09.028176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:09.028176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:09.028176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:09.028176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:09.028176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:09.028176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:09.028176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:09.028176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:09.028176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:09.028176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:09.028805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:09.028805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:09.028805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.028805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:09.029125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.029125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:09.029125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:09.029125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:09.029125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:09.029125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:09.029125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:09.029125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:09.029125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:09.029125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:09.029125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:09.029125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:09.029125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:09.029125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:09.029125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:09.029125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:09.029921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:42:09.029921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:42:09.029921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:42:09.029921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:09.029921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:09.029921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:09.029921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:09.029921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:09.029921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:09.029921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:09.029921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:09.029921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:09.029921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:09.029921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:09.029921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:09.029921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:09.029921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:09.029921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:09.030724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:09.030724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:09.030724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:09.030724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:09.030724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:09.030724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.030724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:09.043212 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:09.043686 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:09.043843 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:09.044020 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:09.044708 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:09.045019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.045839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:09.045839 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:09.045839 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:09.045839 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:09.045839 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:09.045839 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:09.045839 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.045839 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:09.047161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.047161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:42:09.047161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:09.047161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:09.047161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:09.047161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:09.047161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:09.047161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:09.047161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:09.047161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:09.047161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:09.047161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:42:09.047161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:09.047161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:42:09.047161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:42:09.048092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:09.048092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:42:09.048092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:09.048092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:42:09.048092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:42:09.048092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:09.048092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:42:09.048092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.048092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:42:09.048758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.048758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:09.048758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:09.048758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:09.048758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:09.048758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:42:09.048758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:42:09.048758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:09.048758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:09.048758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:09.048758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:09.049306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:09.049306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:09.049306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:09.049306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:09.049306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:09.049306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:09.049306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:09.049306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:09.049306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:09.049306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:09.049306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:09.049306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:09.049306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:09.049306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:09.049697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:09.049697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:09.049697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.049697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:09.050498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.050498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:09.050498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:09.050498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:09.050498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:09.050498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:09.050498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:09.050498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:09.050498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:09.050498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:09.050498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:09.050498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:09.050498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:09.050498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:09.050498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:09.050498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:09.050498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:42:09.051006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:42:09.051006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:09.051006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:09.051006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:09.051006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:09.051006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:09.051006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:09.051006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:09.051006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:09.051006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:09.051006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:09.051006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:09.051006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:09.051006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:09.051006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:09.051006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:09.051006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:09.051006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:09.051006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:09.051655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:09.051655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:09.051655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.051655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:09.116096 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:09.116475 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:09.116648 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:09.116970 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:09.117524 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:09.118040 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:09.118408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.119023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:09.119023 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:09.119023 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:09.119023 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:09.119023 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:09.119023 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:09.119023 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.119023 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:09.120388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.120388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:09.120388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:09.120388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:09.120388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:09.120388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:42:09.120388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:09.120388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:09.120388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:09.120388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:09.120388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:09.120388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:09.120985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:09.120985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:09.120985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:09.120985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:09.120985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:09.120985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:09.120985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:09.120985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:09.120985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:09.120985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:09.120985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:09.120985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:09.120985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:09.121652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.121652 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:09.121993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.121993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:09.121993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:09.121993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:09.121993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:09.121993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:09.121993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:09.121993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:09.121993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:09.121993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:09.121993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:09.121993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:09.121993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:09.121993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:09.121993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:09.121993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:09.122957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:42:09.122957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:09.122957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:09.122957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:09.122957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:09.122957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:09.122957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:09.122957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:09.122957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:09.122957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:09.122957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:09.122957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:09.122957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:09.122957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:09.122957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:09.122957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:09.122957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:09.122957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:09.122957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:09.123783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:09.123783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:09.123783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:13.782420 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:13.783430 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:13.783430 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:13.784589 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:13.816215 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:13.816669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:13.817638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:13.817638 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:13.817638 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:13.817638 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:13.817638 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:13.817638 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:13.817638 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:13.817638 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:13.819352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:13.819352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:13.819352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:13.819352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:13.819352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:13.819352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:42:13.819352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:13.819352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:13.819352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:13.819352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:13.819352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:13.819352 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:13.820067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:13.820067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:13.820067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:13.820067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:13.820067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:13.820067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:13.820067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:13.820067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:13.820067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:13.820067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:13.820067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:13.820067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:13.820067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:13.820816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:13.820816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:13.820816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:13.820816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:13.820816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:13.820816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:13.820816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:13.820816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:13.820816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:13.820816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:13.820816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:13.820816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:13.820816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:13.820816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:13.820816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:13.821641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:13.821641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:13.821641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:13.821641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:42:13.821641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:13.821641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:13.821641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:13.821641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:13.821641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:13.821641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:13.821641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:13.821641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:13.821641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:13.821641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:13.821641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:13.821641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:13.821641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:13.821641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:13.821641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:13.822663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:13.822663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:13.822663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:13.822663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:13.822663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:13.822663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:13.996444 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:13.997189 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:13.997416 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:13.998616 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:13.999068 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:13.999311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.000239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:14.000239 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:14.000239 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:14.000239 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:14.000239 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:14.000239 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:14.000239 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.000239 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:14.002055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.002055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:14.002055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:14.002055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:14.002055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:14.002055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:42:14.002055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:14.002055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:14.002055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:14.002055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:14.002055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:14.002055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:14.002799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:14.002799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:14.002799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:14.002799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:14.002799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:14.002799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:14.002799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:14.002799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:14.002799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:14.002799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:14.002799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:14.002799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:14.002799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:14.003551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.003551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:14.003890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.003890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:14.003890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:14.003890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:14.003890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:14.003890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:14.003890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:14.003890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:14.003890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:14.003890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:14.003890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:14.003890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:14.003890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:14.003890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:14.003890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:14.003890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:14.004635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:42:14.004635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:14.004635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:14.004635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:14.004635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:14.004635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:14.004635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:14.004635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:14.004635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:14.004635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:14.004635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:14.004635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:14.004635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:14.004635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:14.004635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:14.004635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:14.004635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:14.004635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:14.004635 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:14.005557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:14.005557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.005557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:14.022777 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:14.023295 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:14.023539 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:14.023850 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:42:14.024456 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:14.024862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.025726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:14.025726 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:14.025726 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:14.025726 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:14.025726 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:14.025726 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:14.025726 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.025726 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:14.026832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.026832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:42:14.026832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:14.026832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:14.026832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:14.026832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:14.026832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:14.026832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:14.026832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:14.026832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:14.026832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:14.026832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:42:14.026832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:14.026832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:42:14.026832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:42:14.027733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:14.027733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:42:14.027733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:14.027733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:42:14.027733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:42:14.027733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:14.027733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:42:14.027733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.027733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:42:14.028244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.028244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:14.028244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:14.028244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:14.028244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:14.028244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:42:14.028244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:42:14.028244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:42:14.028244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:14.028244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:14.028244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:14.028244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:14.028907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:14.028907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:14.028907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:14.028907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:14.028907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:14.028907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:14.028907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:14.028907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:14.028907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:14.028907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:14.028907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:14.028907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:14.028907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:14.028907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:14.029540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:14.029540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:14.029540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.029540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:14.029540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.029540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:14.029540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:14.029540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:14.029540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:14.029540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:14.029540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:14.029540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:14.029540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:14.029540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:14.030343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:14.030343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:14.030343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:14.030343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:14.030343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:14.030343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:14.030343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:42:14.030343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:42:14.030343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:42:14.030343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:14.030343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:14.030343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:14.030343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:14.030343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:14.030343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:14.030343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:14.030343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:14.030343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:14.030343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:14.031305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:14.031305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:14.031305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:14.031305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:14.031305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:14.031305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:14.031305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:14.031305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:14.031305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:14.031305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:14.031305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.031305 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:14.043741 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:14.044191 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:14.044376 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:14.044588 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:14.045143 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:14.045529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.046351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:14.046351 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:14.046351 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:14.046351 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:14.046351 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:14.046351 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:14.046351 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.046351 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:14.047836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.047836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:42:14.047836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:14.047836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:14.047836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:14.047836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:14.047836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:14.047836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:14.047836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:14.047836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:14.047836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:14.047836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:42:14.047836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:14.047836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:42:14.047836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:42:14.048510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:14.048510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:42:14.048510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:14.048510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:42:14.048510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:42:14.048510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:14.048510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:42:14.048510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.048510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:42:14.049706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.049706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:14.049706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:14.049706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:14.049706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:14.049706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:42:14.049706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:42:14.049706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:14.049706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:14.049706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:14.049706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:14.050418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:14.050418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:14.050418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:14.050418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:14.050418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:14.050418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:14.050418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:14.050418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:14.050418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:14.050418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:14.050418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:14.050418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:14.050418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:14.050418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:14.051172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:14.051172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:14.051172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.051172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:14.051636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.051636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:14.051636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:14.051636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:14.051636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:14.051636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:14.051636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:14.051636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:14.051636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:14.051636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:14.051636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:14.051636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:14.051636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:14.051636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:14.051636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:14.051636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:14.051636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:42:14.052790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:42:14.052790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:14.052790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:14.052790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:14.052790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:14.052790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:14.052790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:14.052790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:14.052790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:14.052790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:14.052790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:14.052790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:14.052790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:14.052790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:14.052790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:14.052790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:14.052790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:14.052790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:14.052790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:14.053780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:14.053780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:14.053780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.053780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:14.117341 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:14.117716 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:14.117958 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:14.118220 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:14.118670 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:14.119141 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:14.119338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.120226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:14.120226 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:14.120226 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:14.120226 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:14.120226 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:14.120226 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:14.120226 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.120226 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:14.122008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.122008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:14.122008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:14.122008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:14.122008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:14.122008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:42:14.122008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:14.122008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:14.122008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:14.122008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:14.122008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:14.122008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:14.122383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:14.122383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:14.122383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:14.122383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:14.122383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:14.122383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:14.122383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:14.122383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:14.122383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:14.122383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:14.122383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:14.122383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:14.122383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:14.122791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.122791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:14.123949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.123949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:14.123949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:14.123949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:14.123949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:14.123949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:14.123949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:14.123949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:14.123949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:14.123949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:14.123949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:14.123949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:14.123949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:14.123949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:14.123949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:14.123949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:14.124411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:42:14.124411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:14.124411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:14.124411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:14.124411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:14.124411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:14.124411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:14.124411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:14.124411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:14.124411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:14.124411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:14.124411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:14.124411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:14.124411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:14.124411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:14.124411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:14.124411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:14.124411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:14.124411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:14.124975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:14.124975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:14.124975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:18.783100 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:18.783652 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:18.783797 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:18.784753 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:18.822809 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:18.822809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:18.823523 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:18.823523 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:18.823523 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:18.823523 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:18.823523 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:18.823523 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:18.823523 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:18.823523 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:18.824541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:18.824541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:18.824541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:18.824541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:18.824541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:18.824541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:42:18.824541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:18.824541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:18.824541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:18.824541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:18.824541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:18.824541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:18.825207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:18.825207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:18.825207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:18.825207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:18.825207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:18.825207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:18.825207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:18.825207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:18.825207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:18.825207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:18.825207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:18.825207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:18.825207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:18.825723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:18.825723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:18.825723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:18.825723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:18.825723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:18.825723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:18.825723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:18.825723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:18.825723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:18.825723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:18.825723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:18.825723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:18.825723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:18.825723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:18.825723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:18.826338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:18.826338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:18.826338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:18.826338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:42:18.826338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:18.826338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:18.826338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:18.826338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:18.826338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:18.826338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:18.826338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:18.826338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:18.826338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:18.826338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:18.826338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:18.826338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:18.826338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:18.826338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:18.826338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:18.827091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:18.827091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:18.827091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:18.827091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:18.827091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:18.827091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:18.997220 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:18.997785 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:18.997969 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:18.998954 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:18.999273 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:18.999518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.000538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:19.000538 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:19.000538 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:19.000538 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:19.000538 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:19.000538 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:19.000538 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.000538 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:19.001643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.001643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:19.001643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:19.001643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:19.001643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:19.001643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:42:19.001643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:19.001643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:19.001643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:19.001643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:19.001643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:19.001643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:19.002377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:19.002377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:19.002377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:19.002377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:19.002377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:19.002377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:19.002377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:19.002377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:19.002377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:19.002377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:19.002377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:19.002377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:19.002377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:19.003153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.003153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:19.003153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.003153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:19.003153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:19.003153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:19.003153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:19.003153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:19.003153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:19.003153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:19.003153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:19.003153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:19.003153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:19.003153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:19.003153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:19.004037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:19.004037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:19.004037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:19.004037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:42:19.004037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:19.004037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:19.004037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:19.004037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:19.004037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:19.004037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:19.004037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:19.004037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:19.004037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:19.004037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:19.004037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:19.004037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:19.004037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:19.004037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:19.004037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:19.004954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:19.004954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:19.004954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:19.004954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:19.004954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.004954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:19.023265 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:19.023720 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:19.023828 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:19.023984 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:42:19.024478 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:19.024798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.025506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:19.025506 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:19.025506 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:19.025506 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:19.025506 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:19.025506 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:19.025506 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.025506 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:19.026419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.026419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:42:19.026419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:19.026419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:19.026419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:19.026419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:19.026419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:19.026419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:19.026419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:19.026419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:19.026419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:19.026419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:42:19.026419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:19.026419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:42:19.026419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:42:19.027199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:19.027199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:42:19.027199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:19.027199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:42:19.027199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:42:19.027199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:19.027199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:42:19.027199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.027199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:42:19.027707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.027707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:19.027707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:19.027707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:19.027707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:19.027707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:42:19.027707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:42:19.027707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:42:19.027707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:19.027707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:19.027707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:19.027707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:19.028377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:19.028377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:19.028377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:19.028377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:19.028377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:19.028377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:19.028377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:19.028377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:19.028377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:19.028377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:19.028377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:19.028377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:19.028377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:19.028377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:19.028889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:19.028889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:19.028889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.028889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:19.028889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.028889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:19.028889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:19.028889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:19.028889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:19.028889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:19.028889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:19.028889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:19.028889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:19.028889 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:19.029257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:19.029257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:19.029257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:19.029257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:19.029257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:19.029257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:19.029257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:42:19.029257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:42:19.029257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:42:19.029257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:19.029257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:19.029257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:19.029257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:19.029257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:19.029257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:19.029257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:19.029257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:19.029257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:19.029257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:19.029913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:19.029913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:19.029913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:19.029913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:19.029913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:19.029913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:19.029913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:19.029913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:19.029913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:19.029913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:19.029913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.029913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:19.044323 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:19.044702 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:19.044796 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:19.044957 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:19.045452 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:19.045917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.046643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:19.046643 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:19.046643 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:19.046643 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:19.046643 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:19.046643 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:19.046643 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.046643 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:19.047542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.047542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:42:19.047542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:19.047542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:19.047542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:19.047542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:19.047542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:19.047542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:19.047542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:19.047542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:19.047542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:19.047542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:42:19.047542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:19.047542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:42:19.047542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:42:19.048419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:19.048419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:42:19.048419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:19.048419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:42:19.048419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:42:19.048419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:19.048419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:42:19.048419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.048419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:42:19.048879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.048879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:19.048879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:19.048879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:19.048879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:19.048879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:42:19.048879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:42:19.048879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:19.048879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:19.048879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:19.048879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:19.049420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:19.049420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:19.049420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:19.049420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:19.049420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:19.049420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:19.049420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:19.049420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:19.049420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:19.049420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:19.049420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:19.049420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:19.049420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:19.049420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:19.050093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:19.050093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:19.050093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.050093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:19.050093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.050093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:19.050093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:19.050093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:19.050093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:19.050093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:19.050093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:19.050093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:19.050093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:19.050093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:19.051211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:19.051211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:19.051211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:19.051211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:19.051211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:19.051211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:19.051211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:42:19.051211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:42:19.051211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:19.051211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:19.051211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:19.051211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:19.051211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:19.051211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:19.051211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:19.051211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:19.051211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:19.051211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:19.051211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:19.052210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:19.052210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:19.052210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:19.052210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:19.052210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:19.052210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:19.052210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:19.052210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:19.052210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:19.052210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.052210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:19.117813 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:19.118341 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:19.118438 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:19.118662 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:19.119183 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:19.119568 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:19.119839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.120774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:19.120774 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:19.120774 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:19.120774 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:19.120774 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:19.120774 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:19.120774 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.120774 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:19.121877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.121877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:19.121877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:19.121877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:19.121877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:19.121877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:42:19.121877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:19.121877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:19.121877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:19.121877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:19.121877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:19.121877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:19.122771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:19.122771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:19.122771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:19.122771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:19.122771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:19.122771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:19.122771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:19.122771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:19.122771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:19.122771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:19.122771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:19.122771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:19.122771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:19.123387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.123387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:19.123387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.123387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:19.123387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:19.123387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:19.123387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:19.123387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:19.123387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:19.123387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:19.123387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:19.123387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:19.123387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:19.123387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:19.123387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:19.124208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:19.124208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:19.124208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:19.124208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:42:19.124208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:19.124208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:19.124208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:19.124208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:19.124208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:19.124208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:19.124208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:19.124208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:19.124208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:19.124208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:19.124208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:19.124208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:19.124208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:19.124208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:19.124208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:19.125164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:19.125164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:19.125164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:19.125164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:19.125164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:19.125164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:23.783661 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:23.784304 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:23.784304 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:23.785229 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:23.814823 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:23.815063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:23.815802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:23.815802 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:23.815802 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:23.815802 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:23.815802 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:23.815802 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:23.815802 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:23.815802 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:23.816649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:23.816649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:23.816649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:23.816649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:23.816649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:23.816649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:42:23.816649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:23.816649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:23.816649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:23.816649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:23.816649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:23.816649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:23.817328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:23.817328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:23.817328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:23.817328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:23.817328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:23.817328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:23.817328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:23.817328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:23.817328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:23.817328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:23.817328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:23.817328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:23.817328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:23.818054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:23.818054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:23.818054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:23.818054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:23.818054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:23.818054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:23.818054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:23.818054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:23.818054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:23.818054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:23.818054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:23.818054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:23.818054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:23.818054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:23.818054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:23.818904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:23.818904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:23.818904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:23.818904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:42:23.818904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:23.818904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:23.818904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:23.818904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:23.818904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:23.818904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:23.818904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:23.818904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:23.818904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:23.818904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:23.818904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:23.818904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:23.818904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:23.818904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:23.818904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:23.820188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:23.820188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:23.820188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:23.820188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:23.820188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:23.820188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:23.997843 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:23.998464 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:23.998632 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:23.999863 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:24.000236 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:24.000490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.001378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:24.001378 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:24.001378 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:24.001378 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:24.001378 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:24.001378 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:24.001378 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.001378 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:24.002914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.002914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:24.002914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:24.002914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:24.002914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:24.002914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:42:24.002914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:24.002914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:24.002914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:24.002914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:24.002914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:24.002914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:24.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:24.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:24.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:24.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:24.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:24.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:24.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:24.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:24.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:24.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:24.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:24.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:24.003289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:24.003661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.003661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:24.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:24.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:24.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:24.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:24.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:24.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:24.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:24.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:24.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:24.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:24.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:24.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:24.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:24.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:24.003943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:24.004374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:42:24.004374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:24.004374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:24.004374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:24.004374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:24.004374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:24.004374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:24.004374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:24.004374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:24.004374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:24.004374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:24.004374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:24.004374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:24.004374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:24.004374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:24.004374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:24.004374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:24.004374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:24.004374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:24.004919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:24.004919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.004919 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:24.023640 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:24.024007 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:24.024172 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:24.024358 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:42:24.024851 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:24.025058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.025945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:24.025945 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:24.025945 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:24.025945 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:24.025945 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:24.025945 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:24.025945 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.025945 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:24.027203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.027203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:42:24.027203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:24.027203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:24.027203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:24.027203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:24.027203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:24.027203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:24.027203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:24.027203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:24.027203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:24.027203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:42:24.027203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:24.027203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:42:24.027203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:42:24.027947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:24.027947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:42:24.027947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:24.027947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:42:24.027947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:42:24.027947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:24.027947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:42:24.027947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.027947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:42:24.028537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.028537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:24.028537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:24.028537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:24.028537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:24.028537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:42:24.028537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:42:24.028537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:42:24.028537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:24.028537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:24.028537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:24.028537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:24.029110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:24.029110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:24.029110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:24.029110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:24.029110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:24.029110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:24.029110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:24.029110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:24.029110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:24.029110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:24.029110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:24.029110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:24.029110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:24.029110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:24.029769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:24.029769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:24.029769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.029769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:24.030508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.030508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:24.030508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:24.030508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:24.030508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:24.030508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:24.030508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:24.030508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:24.030508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:24.030508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:24.030508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:24.030508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:24.030508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:24.030508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:24.030508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:24.030508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:24.031320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:42:24.031320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:42:24.031320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:42:24.031320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:24.031320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:24.031320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:24.031320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:24.031320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:24.031320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:24.031320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:24.031320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:24.031320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:24.031320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:24.031320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:24.031320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:24.031320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:24.031320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:24.031320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:24.032212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:24.032212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:24.032212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:24.032212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:24.032212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:24.032212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.032212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:24.044692 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:24.045022 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:24.045382 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:24.045382 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:24.045808 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:24.046056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.046697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:24.046697 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:24.046697 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:24.046697 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:24.046697 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:24.046697 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:24.046697 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.046697 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:24.047920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.047920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:42:24.047920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:24.047920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:24.047920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:24.047920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:24.047920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:24.047920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:24.047920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:24.047920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:24.047920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:24.047920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:42:24.047920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:24.047920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:42:24.047920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:42:24.048561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:24.048561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:42:24.048561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:24.048561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:42:24.048561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:42:24.048561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:24.048561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:42:24.048561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.048561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:42:24.049326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.049326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:24.049326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:24.049326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:24.049326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:24.049326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:42:24.049326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:42:24.049326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:24.049326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:24.049326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:24.049326 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:24.049924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:24.049924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:24.049924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:24.049924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:24.049924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:24.049924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:24.049924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:24.049924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:24.049924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:24.049924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:24.049924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:24.049924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:24.049924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:24.049924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:24.050508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:24.050508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:24.050508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.050508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:24.051242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.051242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:24.051242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:24.051242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:24.051242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:24.051242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:24.051242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:24.051242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:24.051242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:24.051242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:24.051242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:24.051242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:24.051242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:24.051242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:24.051242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:24.051242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:24.051242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:42:24.052114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:42:24.052114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:24.052114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:24.052114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:24.052114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:24.052114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:24.052114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:24.052114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:24.052114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:24.052114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:24.052114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:24.052114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:24.052114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:24.052114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:24.052114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:24.052114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:24.052114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:24.052114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:24.052114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:24.053056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:24.053056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:24.053056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.053056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:24.118235 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:24.118690 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:24.119088 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:24.119244 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:24.119638 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:24.119927 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:24.120359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.120776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:24.120776 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:24.120776 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:24.120776 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:24.120776 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:24.120776 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:24.120776 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.120776 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:24.122289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.122289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:24.122289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:24.122289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:24.122289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:24.122289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:42:24.122289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:24.122289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:24.122289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:24.122289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:24.122289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:24.122289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:24.123198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:24.123198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:24.123198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:24.123198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:24.123198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:24.123198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:24.123198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:24.123198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:24.123198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:24.123198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:24.123198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:24.123198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:24.123198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:24.123830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.123830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:24.123830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.123830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:24.123830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:24.123830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:24.123830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:24.123830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:24.123830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:24.123830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:24.123830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:24.123830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:24.123830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:24.123830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:24.123830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:24.124585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:24.124585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:24.124585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:24.124585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:42:24.124585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:24.124585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:24.124585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:24.124585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:24.124585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:24.124585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:24.124585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:24.124585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:24.124585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:24.124585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:24.124585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:24.124585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:24.124585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:24.124585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:24.124585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:24.125490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:24.125490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:24.125490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:24.125490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:24.125490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:24.125490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:28.784351 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:28.785270 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:28.785270 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:28.786298 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:28.819515 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:28.819914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:28.820703 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:28.820703 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:28.820703 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:28.820703 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:28.820703 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:28.820703 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:28.820703 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:28.820703 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:28.821460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:28.821460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:28.821460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:28.821460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:28.821460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:28.821460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:42:28.821460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:28.821460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:28.821460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:28.821460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:28.821460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:28.821460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:28.822188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:28.822188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:28.822188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:28.822188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:28.822188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:28.822188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:28.822188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:28.822188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:28.822188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:28.822188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:28.822188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:28.822188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:28.822188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:28.823117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:28.823117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:28.823117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:28.823117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:28.823117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:28.823117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:28.823117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:28.823117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:28.823117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:28.823117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:28.823117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:28.823117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:28.823117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:28.823117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:28.823117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:28.824173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:28.824173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:28.824173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:28.824173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:42:28.824173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:28.824173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:28.824173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:28.824173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:28.824173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:28.824173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:28.824173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:28.824173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:28.824173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:28.824173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:28.824173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:28.824173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:28.824173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:28.824173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:28.824173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:28.825956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:28.825956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:28.825956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:28.825956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:28.825956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:28.825956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:28.998509 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:28.999222 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:28.999443 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:29.000653 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:29.001065 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:29.001298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.002263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:29.002263 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:29.002263 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:29.002263 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:29.002263 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:29.002263 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:29.002263 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.002263 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:29.004002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.004002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:29.004002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:29.004002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:29.004002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:29.004002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:42:29.004002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:29.004002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:29.004002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:29.004002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:29.004002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:29.004002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:29.004775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:29.004775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:29.004775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:29.004775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:29.004775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:29.004775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:29.004775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:29.004775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:29.004775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:29.004775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:29.004775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:29.004775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:29.004775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:29.005438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.005438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:29.006249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.006249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:29.006249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:29.006249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:29.006249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:29.006249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:29.006249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:29.006249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:29.006249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:29.006249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:29.006249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:29.006249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:29.006249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:29.006249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:29.006249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:29.006249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:29.007136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:42:29.007136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:29.007136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:29.007136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:29.007136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:29.007136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:29.007136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:29.007136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:29.007136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:29.007136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:29.007136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:29.007136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:29.007136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:29.007136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:29.007136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:29.007136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:29.007136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:29.007136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:29.007136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:29.008039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:29.008039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.008039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:29.024074 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:29.024432 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:29.024556 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:29.024803 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:42:29.025290 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:29.025599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.026463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:29.026463 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:29.026463 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:29.026463 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:29.026463 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:29.026463 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:29.026463 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.026463 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:29.027885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.027885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:42:29.027885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:29.027885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:29.027885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:29.027885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:29.027885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:29.027885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:29.027885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:29.027885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:29.027885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:29.027885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:42:29.027885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:29.027885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:42:29.027885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:42:29.028787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:29.028787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:42:29.028787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:29.028787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:42:29.028787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:42:29.028787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:29.028787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:42:29.028787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.028787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:42:29.029624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.029624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:29.029624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:29.029624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:29.029624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:29.029624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:42:29.029624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:42:29.029624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:42:29.029624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:29.029624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:29.029624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:29.029624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:29.030215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:29.030215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:29.030215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:29.030215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:29.030215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:29.030215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:29.030215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:29.030215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:29.030215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:29.030215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:29.030215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:29.030215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:29.030215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:29.030215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:29.030803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:29.030803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:29.030803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.030803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:29.031636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.031636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:29.031636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:29.031636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:29.031636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:29.031636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:29.031636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:29.031636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:29.031636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:29.031636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:29.031636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:29.031636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:29.031636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:29.031636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:29.031636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:29.031636 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:29.032349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:42:29.032349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:42:29.032349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:42:29.032349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:29.032349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:29.032349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:29.032349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:29.032349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:29.032349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:29.032349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:29.032349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:29.032349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:29.032349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:29.032349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:29.032349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:29.032349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:29.032349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:29.032349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:29.033218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:29.033218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:29.033218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:29.033218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:29.033218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:29.033218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.033218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:29.045166 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:29.045485 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:29.045753 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:29.046140 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:29.046530 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:29.046900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.047640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:29.047640 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:29.047640 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:29.047640 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:29.047640 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:29.047640 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:29.047640 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.047640 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:29.048948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.048948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:42:29.048948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:29.048948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:29.048948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:29.048948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:29.048948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:29.048948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:29.048948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:29.048948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:29.048948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:29.048948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:42:29.048948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:29.048948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:42:29.048948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:42:29.049753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:29.049753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:42:29.049753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:29.049753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:42:29.049753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:42:29.049753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:29.049753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:42:29.049753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.049753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:42:29.050515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.050515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:29.050515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:29.050515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:29.050515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:29.050515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:42:29.050515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:42:29.050515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:29.050515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:29.050515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:29.050515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:29.051124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:29.051124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:29.051124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:29.051124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:29.051124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:29.051124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:29.051124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:29.051124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:29.051124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:29.051124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:29.051124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:29.051124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:29.051124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:29.051124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:29.051810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:29.051810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:29.051810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.051810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:29.052011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.052011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:29.052011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:29.052011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:29.052011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:29.052011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:29.052011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:29.052011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:29.052011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:29.052011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:29.052011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:29.052011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:29.052011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:29.052011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:29.052011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:29.052011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:29.052011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:42:29.052885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:42:29.052885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:29.052885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:29.052885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:29.052885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:29.052885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:29.052885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:29.052885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:29.052885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:29.052885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:29.052885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:29.052885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:29.052885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:29.052885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:29.052885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:29.052885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:29.052885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:29.052885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:29.052885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:29.053827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:29.053827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:29.053827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.053827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:29.118738 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:29.119139 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:29.119411 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:29.119822 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:29.120433 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:29.120930 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:29.121316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.122244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:29.122244 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:29.122244 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:29.122244 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:29.122244 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:29.122244 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:29.122244 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.122244 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:29.123839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.123839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:29.123839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:29.123839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:29.123839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:29.123839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:42:29.123839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:29.123839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:29.123839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:29.123839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:29.123839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:29.123839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:29.124478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:29.124478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:29.124478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:29.124478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:29.124478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:29.124478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:29.124478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:29.124478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:29.124478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:29.124478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:29.124478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:29.124478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:29.124478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:29.125229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.125229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:29.126022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.126022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:29.126022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:29.126022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:29.126022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:29.126022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:29.126022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:29.126022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:29.126022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:29.126022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:29.126022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:29.126022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:29.126022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:29.126022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:29.126022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:29.126022 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:29.126496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:42:29.126496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:29.126496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:29.126496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:29.126496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:29.126496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:29.126496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:29.126496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:29.126496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:29.126496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:29.126496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:29.126496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:29.126496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:29.126496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:29.126496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:29.126496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:29.126496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:29.126496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:29.126496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:29.127039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:29.127039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:29.127039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:33.784981 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:33.785755 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:33.785843 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:33.787072 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:33.820661 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:33.821208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:33.822106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:33.822106 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:33.822106 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:33.822106 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:33.822106 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:33.822106 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:33.822106 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:33.822106 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:33.823030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:33.823030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:33.823030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:33.823030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:33.823030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:33.823030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:42:33.823030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:33.823030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:33.823030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:33.823030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:33.823030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:33.823030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:33.823634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:33.823634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:33.823634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:33.823634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:33.823634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:33.823634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:33.823634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:33.823634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:33.823634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:33.823634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:33.823634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:33.823634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:33.823634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:33.824283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:33.824283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:33.824283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:33.824283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:33.824283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:33.824283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:33.824283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:33.824283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:33.824283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:33.824283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:33.824283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:33.824283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:33.824283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:33.824283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:33.824283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:33.825181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:33.825181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:33.825181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:33.825181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:42:33.825181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:33.825181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:33.825181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:33.825181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:33.825181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:33.825181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:33.825181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:33.825181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:33.825181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:33.825181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:33.825181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:33.825181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:33.825181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:33.825181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:33.825181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:33.826234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:33.826234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:33.826234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:33.826234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:33.826234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:33.826234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:33.999211 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:33.999902 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:34.000097 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:34.001124 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:34.001493 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:34.001704 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.002580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:34.002580 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:34.002580 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:34.002580 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:34.002580 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:34.002580 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:34.002580 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.002580 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:34.004214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.004214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:34.004214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:34.004214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:34.004214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:34.004214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:42:34.004214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:34.004214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:34.004214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:34.004214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:34.004214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:34.004214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:34.004827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:34.004827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:34.004827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:34.004827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:34.004827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:34.004827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:34.004827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:34.004827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:34.004827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:34.004827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:34.004827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:34.004827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:34.004827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:34.005378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.005378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:34.005378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.005378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:34.005378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:34.005378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:34.005378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:34.005378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:34.005378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:34.005378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:34.005378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:34.005378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:34.005378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:34.005378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:34.005378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:34.006124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:34.006124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:34.006124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:34.006124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:42:34.006124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:34.006124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:34.006124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:34.006124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:34.006124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:34.006124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:34.006124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:34.006124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:34.006124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:34.006124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:34.006124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:34.006124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:34.006124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:34.006124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:34.006124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:34.006945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:34.006945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:34.006945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:34.006945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:34.006945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.006945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:34.024638 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:34.025136 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:34.025308 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:34.025491 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:42:34.026049 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:34.026347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.027140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:34.027140 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:34.027140 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:34.027140 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:34.027140 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:34.027140 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:34.027140 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.027140 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:34.028382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.028382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:42:34.028382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:34.028382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:34.028382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:34.028382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:34.028382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:34.028382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:34.028382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:34.028382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:34.028382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:34.028382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:42:34.028382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:34.028382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:42:34.028382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:42:34.029128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:34.029128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:42:34.029128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:34.029128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:42:34.029128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:42:34.029128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:34.029128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:42:34.029128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.029128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:42:34.029808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.029808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:34.029808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:34.029808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:34.029808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:34.029808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:42:34.029808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:42:34.029808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:42:34.029808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:34.029808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:34.029808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:34.029808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:34.030378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:34.030378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:34.030378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:34.030378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:34.030378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:34.030378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:34.030378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:34.030378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:34.030378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:34.030378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:34.030378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:34.030378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:34.030378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:34.030378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:34.031136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:34.031136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:34.031136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.031136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:34.031409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.031409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:34.031409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:34.031409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:34.031409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:34.031409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:34.031409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:34.031409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:34.031409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:34.031409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:34.031409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:34.031409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:34.031409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:34.031409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:34.031409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:34.031409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:34.032357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:42:34.032357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:42:34.032357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:42:34.032357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:34.032357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:34.032357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:34.032357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:34.032357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:34.032357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:34.032357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:34.032357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:34.032357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:34.032357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:34.032357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:34.032357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:34.032357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:34.032357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:34.032357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:34.032900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:34.032900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:34.032900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:34.032900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:34.032900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:34.032900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.032900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:34.045547 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:34.045811 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:34.045988 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:34.046176 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:34.046791 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:34.047256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.048187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:34.048187 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:34.048187 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:34.048187 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:34.048187 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:34.048187 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:34.048187 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.048187 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:34.049403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.049403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:42:34.049403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:34.049403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:34.049403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:34.049403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:34.049403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:34.049403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:34.049403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:34.049403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:34.049403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:34.049403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:42:34.049403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:34.049403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:42:34.049403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:42:34.050116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:34.050116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:42:34.050116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:34.050116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:42:34.050116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:42:34.050116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:34.050116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:42:34.050116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.050116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:42:34.050882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.050882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:34.050882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:34.050882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:34.050882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:34.050882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:42:34.050882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:42:34.050882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:34.050882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:34.050882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:34.050882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:34.051456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:34.051456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:34.051456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:34.051456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:34.051456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:34.051456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:34.051456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:34.051456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:34.051456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:34.051456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:34.051456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:34.051456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:34.051456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:34.051456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:34.052161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:34.052161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:34.052161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.052161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:34.052702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.052702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:34.052702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:34.052702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:34.052702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:34.052702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:34.052702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:34.052702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:34.052702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:34.052702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:34.052702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:34.052702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:34.052702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:34.052702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:34.052702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:34.052702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:34.052702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:42:34.053657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:42:34.053657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:34.053657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:34.053657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:34.053657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:34.053657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:34.053657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:34.053657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:34.053657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:34.053657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:34.053657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:34.053657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:34.053657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:34.053657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:34.053657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:34.053657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:34.053657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:34.053657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:34.053657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:34.054958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:34.054958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:34.054958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.054958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:34.119205 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:34.119790 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:34.119990 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:34.120240 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:34.120938 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:34.121255 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:34.121484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.122371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:34.122371 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:34.122371 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:34.122371 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:34.122371 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:34.122371 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:34.122371 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.122371 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:34.123835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.123835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:34.123835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:34.123835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:34.123835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:34.123835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:42:34.123835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:34.123835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:34.123835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:34.123835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:34.123835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:34.123835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:34.124443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:34.124443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:34.124443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:34.124443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:34.124443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:34.124443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:34.124443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:34.124443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:34.124443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:34.124443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:34.124443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:34.124443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:34.124443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:34.125011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.125011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:34.125782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.125782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:34.125782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:34.125782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:34.125782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:34.125782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:34.125782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:34.125782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:34.125782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:34.125782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:34.125782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:34.125782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:34.125782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:34.125782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:34.125782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:34.125782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:34.126373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:42:34.126373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:34.126373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:34.126373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:34.126373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:34.126373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:34.126373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:34.126373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:34.126373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:34.126373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:34.126373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:34.126373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:34.126373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:34.126373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:34.126373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:34.126373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:34.126373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:34.126373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:34.126373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:34.126953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:34.126953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:34.126953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:38.785658 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:38.786487 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:38.786487 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:38.787798 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:38.823432 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:38.824038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:38.825473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:38.825473 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:38.825473 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:38.825473 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:38.825473 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:38.825473 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:38.825473 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:38.825473 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:38.826378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:38.826378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:38.826378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:38.826378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:38.826378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:38.826378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:42:38.826378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:38.826378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:38.826378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:38.826378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:38.826378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:38.826378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:38.827515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:38.827515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:38.827515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:38.827515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:38.827515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:38.827515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:38.827515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:38.827515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:38.827515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:38.827515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:38.827515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:38.827515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:38.827515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:38.828123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:38.828123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:38.828123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:38.828123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:38.828123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:38.828123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:38.828123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:38.828123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:38.828123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:38.828123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:38.828123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:38.828123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:38.828123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:38.828123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:38.828123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:38.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:38.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:38.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:38.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:42:38.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:38.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:38.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:38.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:38.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:38.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:38.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:38.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:38.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:38.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:38.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:38.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:38.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:38.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:38.828883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:38.830569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:38.830569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:38.830569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:38.830569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:38.830569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:38.830569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:38.999962 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:39.000629 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:39.001052 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:39.001884 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:39.002093 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:39.002433 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.002975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:39.002975 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:39.002975 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:39.002975 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:39.002975 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:39.002975 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:39.002975 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.002975 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:39.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:39.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:39.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:39.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:39.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:42:39.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:39.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:39.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:39.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:39.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:39.004509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:39.006737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:39.006737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:39.006737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:39.006737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:39.006737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:39.006737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:39.006737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:39.006737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:39.006737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:39.006737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:39.006737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:39.006737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:39.006737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:39.007484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.007484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:39.007484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.007484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:39.007484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:39.007484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:39.007484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:39.007484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:39.007484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:39.007484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:39.007484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:39.007484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:39.007484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:39.007484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:39.007484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:39.008056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:39.008056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:39.008056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:39.008056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:42:39.008056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:39.008056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:39.008056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:39.008056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:39.008056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:39.008056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:39.008056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:39.008056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:39.008056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:39.008056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:39.008056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:39.008056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:39.008056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:39.008056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:39.008056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:39.008784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:39.008784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:39.008784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:39.008784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:39.008784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.008784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:39.025299 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:39.026004 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:39.026286 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:39.026557 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:42:39.027054 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:39.027263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.028027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:39.028027 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:39.028027 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:39.028027 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:39.028027 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:39.028027 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:39.028027 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.028027 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:39.029234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.029234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:42:39.029234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:39.029234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:39.029234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:39.029234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:39.029234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:39.029234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:39.029234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:39.029234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:39.029234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:39.029234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:42:39.029234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:39.029234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:42:39.029234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:42:39.029725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:39.029725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:42:39.029725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:39.029725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:42:39.029725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:42:39.029725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:39.029725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:42:39.029725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.029725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:42:39.030585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.030585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:39.030585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:39.030585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:39.030585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:39.030585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:42:39.030585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:42:39.030585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:42:39.030585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:39.030585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:39.030585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:39.030585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:39.031014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:39.031014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:39.031014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:39.031014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:39.031014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:39.031014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:39.031014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:39.031014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:39.031014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:39.031014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:39.031014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:39.031014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:39.031014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:39.031014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:39.031455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:39.031455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:39.031455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.031455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:39.033639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.033639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:39.033639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:39.033639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:39.033639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:39.033639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:39.033639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:39.033639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:39.033639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:39.033639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:39.033639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:39.033639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:39.033639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:39.033639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:39.033639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:39.033639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:39.034182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:42:39.034182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:42:39.034182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:42:39.034182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:39.034182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:39.034182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:39.034182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:39.034182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:39.034182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:39.034182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:39.034182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:39.034182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:39.034182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:39.034182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:39.034182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:39.034182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:39.034182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:39.034182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:39.034903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:39.034903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:39.034903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:39.034903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:39.034903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:39.034903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.034903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:39.045953 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:39.046608 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:39.046910 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:39.047202 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:39.047773 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:39.048155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.048937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:39.048937 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:39.048937 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:39.048937 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:39.048937 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:39.048937 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:39.048937 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.048937 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:39.050262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.050262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:42:39.050262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:39.050262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:39.050262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:39.050262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:39.050262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:39.050262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:39.050262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:39.050262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:39.050262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:39.050262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:42:39.050262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:39.050262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:42:39.050262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:42:39.051416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:39.051416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:42:39.051416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:39.051416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:42:39.051416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:42:39.051416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:39.051416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:42:39.051416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.051416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:42:39.051667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.051667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:39.051667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:39.051667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:39.051667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:39.051667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:42:39.051667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:42:39.051667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:39.051667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:39.051667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:39.051667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:39.051999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:39.051999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:39.051999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:39.051999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:39.051999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:39.051999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:39.051999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:39.051999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:39.051999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:39.051999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:39.051999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:39.051999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:39.051999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:39.051999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:39.052356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:39.052356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:39.052356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.052356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:39.053327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.053327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:39.053327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:39.053327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:39.053327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:39.053327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:39.053327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:39.053327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:39.053327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:39.053327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:39.053327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:39.053327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:39.053327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:39.053327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:39.053327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:39.053327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:39.053327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:42:39.053731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:42:39.053731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:39.053731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:39.053731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:39.053731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:39.053731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:39.053731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:39.053731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:39.053731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:39.053731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:39.053731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:39.053731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:39.053731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:39.053731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:39.053731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:39.053731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:39.053731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:39.053731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:39.053731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:39.054234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:39.054234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:39.054234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.054234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:39.119979 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:39.120490 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:39.120607 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:39.120914 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:39.121420 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:39.121880 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:39.122148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.122980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:39.122980 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:39.122980 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:39.122980 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:39.122980 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:39.122980 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:39.122980 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.122980 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:39.124279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.124279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:39.124279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:39.124279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:39.124279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:39.124279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:42:39.124279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:39.124279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:39.124279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:39.124279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:39.124279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:39.124279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:39.124903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:39.124903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:39.124903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:39.124903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:39.124903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:39.124903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:39.124903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:39.124903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:39.124903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:39.124903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:39.124903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:39.124903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:39.124903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:39.125461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.125461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:39.125865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.125865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:39.125865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:39.125865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:39.125865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:39.125865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:39.125865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:39.125865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:39.125865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:39.125865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:39.125865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:39.125865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:39.125865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:39.125865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:39.125865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:39.125865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:39.126740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:42:39.126740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:39.126740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:39.126740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:39.126740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:39.126740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:39.126740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:39.126740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:39.126740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:39.126740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:39.126740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:39.126740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:39.126740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:39.126740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:39.126740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:39.126740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:39.126740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:39.126740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:39.126740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:39.127615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:39.127615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:39.127615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:43.786410 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:43.787037 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:43.787244 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:43.788553 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:43.818847 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:43.819073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:43.819628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:43.819628 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:43.819628 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:43.819628 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:43.819628 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:43.819628 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:43.819628 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:43.819628 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:43.820612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:43.820612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:43.820612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:43.820612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:43.820612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:43.820612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:42:43.820612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:43.820612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:43.820612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:43.820612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:43.820612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:43.820612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:43.821397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:43.821397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:43.821397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:43.821397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:43.821397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:43.821397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:43.821397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:43.821397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:43.821397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:43.821397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:43.821397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:43.821397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:43.821397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:43.822110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:43.822110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:43.822110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:43.822110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:43.822110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:43.822110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:43.822110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:43.822110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:43.822110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:43.822110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:43.822110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:43.822110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:43.822110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:43.822110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:43.822110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:43.822935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:43.822935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:43.822935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:43.822935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:42:43.822935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:43.822935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:43.822935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:43.822935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:43.822935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:43.822935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:43.822935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:43.822935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:43.822935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:43.822935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:43.822935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:43.822935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:43.822935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:43.822935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:43.822935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:43.823951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:43.823951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:43.823951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:43.823951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:43.823951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:43.823951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:44.000733 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:44.001354 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:44.001613 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:44.002901 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:44.003293 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:44.003544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.004420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:44.004420 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:44.004420 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:44.004420 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:44.004420 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:44.004420 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:44.004420 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.004420 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:44.005634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.005634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:44.005634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:44.005634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:44.005634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:44.005634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:42:44.005634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:44.005634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:44.005634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:44.005634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:44.005634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:44.005634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:44.006201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:44.006201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:44.006201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:44.006201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:44.006201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:44.006201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:44.006201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:44.006201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:44.006201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:44.006201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:44.006201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:44.006201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:44.006201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:44.006824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.006824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:44.007170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.007170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:44.007170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:44.007170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:44.007170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:44.007170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:44.007170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:44.007170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:44.007170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:44.007170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:44.007170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:44.007170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:44.007170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:44.007170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:44.007170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:44.007170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:44.007653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:42:44.007653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:44.007653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:44.007653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:44.007653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:44.007653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:44.007653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:44.007653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:44.007653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:44.007653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:44.007653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:44.007653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:44.007653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:44.007653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:44.007653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:44.007653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:44.007653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:44.007653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:44.007653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:44.008248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:44.008248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.008248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:44.025980 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:44.026435 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:44.026711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:44.026980 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:42:44.027426 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:44.027622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.028343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:44.028343 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:44.028343 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:44.028343 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:44.028343 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:44.028343 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:44.028343 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.028343 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:44.029793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.029793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:42:44.029793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:44.029793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:44.029793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:44.029793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:44.029793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:44.029793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:44.029793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:44.029793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:44.029793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:44.029793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:42:44.029793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:44.029793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:42:44.029793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:42:44.030976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:44.030976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:42:44.030976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:44.030976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:42:44.030976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:42:44.030976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:44.030976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:42:44.030976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.030976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:42:44.031750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.031750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:44.031750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:44.031750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:44.031750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:44.031750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:42:44.031750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:42:44.031750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:42:44.031750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:44.031750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:44.031750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:44.031750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:44.032620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:44.032620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:44.032620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:44.032620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:44.032620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:44.032620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:44.032620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:44.032620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:44.032620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:44.032620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:44.032620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:44.032620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:44.032620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:44.032620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:44.034187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:44.034187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:44.034187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.034187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:44.034187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.034187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:44.034187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:44.034187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:44.034187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:44.034187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:44.034187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:44.034187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:44.034187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:44.034187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:44.034824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:44.034824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:44.034824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:44.034824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:44.034824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:44.034824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:44.034824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:42:44.034824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:42:44.034824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:42:44.034824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:44.034824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:44.034824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:44.034824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:44.034824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:44.034824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:44.034824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:44.034824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:44.034824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:44.034824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:44.035643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:44.035643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:44.035643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:44.035643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:44.035643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:44.035643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:44.035643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:44.035643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:44.035643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:44.035643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:44.035643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.035643 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:44.046734 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:44.047035 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:44.047182 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:44.047414 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:44.047696 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:44.047879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.048441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:44.048441 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:44.048441 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:44.048441 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:44.048441 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:44.048441 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:44.048441 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.048441 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:44.049956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.049956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:42:44.049956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:44.049956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:44.049956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:44.049956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:44.049956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:44.049956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:44.049956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:44.049956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:44.049956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:44.049956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:42:44.049956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:44.049956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:42:44.049956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:42:44.050964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:44.050964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:42:44.050964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:44.050964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:42:44.050964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:42:44.050964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:44.050964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:42:44.050964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.050964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:42:44.051997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.051997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:44.051997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:44.051997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:44.051997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:44.051997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:42:44.051997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:42:44.051997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:44.051997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:44.051997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:44.051997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:44.052785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:44.052785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:44.052785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:44.052785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:44.052785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:44.052785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:44.052785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:44.052785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:44.052785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:44.052785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:44.052785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:44.052785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:44.052785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:44.052785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:44.053998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:44.053998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:44.053998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.053998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:44.054513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.054513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:44.054513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:44.054513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:44.054513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:44.054513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:44.054513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:44.054513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:44.054513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:44.054513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:44.054513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:44.054513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:44.054513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:44.054513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:44.054513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:44.054513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:44.054513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:42:44.055373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:42:44.055373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:44.055373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:44.055373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:44.055373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:44.055373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:44.055373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:44.055373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:44.055373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:44.055373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:44.055373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:44.055373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:44.055373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:44.055373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:44.055373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:44.055373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:44.055373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:44.055373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:44.055373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:44.055991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:44.055991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:44.055991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.055991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:44.120542 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:44.121044 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:44.121250 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:44.121506 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:44.122056 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:44.122408 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:44.122634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.123427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:44.123427 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:44.123427 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:44.123427 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:44.123427 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:44.123427 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:44.123427 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.123427 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:44.124948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.124948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:44.124948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:44.124948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:44.124948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:44.124948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:42:44.124948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:44.124948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:44.124948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:44.124948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:44.124948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:44.124948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:44.125315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:44.125315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:44.125315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:44.125315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:44.125315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:44.125315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:44.125315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:44.125315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:44.125315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:44.125315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:44.125315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:44.125315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:44.125315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:44.125691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.125691 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:44.126637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.126637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:44.126637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:44.126637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:44.126637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:44.126637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:44.126637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:44.126637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:44.126637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:44.126637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:44.126637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:44.126637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:44.126637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:44.126637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:44.126637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:44.126637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:44.127107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:42:44.127107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:44.127107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:44.127107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:44.127107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:44.127107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:44.127107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:44.127107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:44.127107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:44.127107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:44.127107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:44.127107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:44.127107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:44.127107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:44.127107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:44.127107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:44.127107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:44.127107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:44.127107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:44.127591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:44.127591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:44.127591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:48.787136 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:48.788154 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:48.788154 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:48.789258 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:48.824316 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:48.824712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:48.825603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:48.825603 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:48.825603 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:48.825603 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:48.825603 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:48.825603 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:48.825603 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:48.825603 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:48.826597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:48.826597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:48.826597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:48.826597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:48.826597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:48.826597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:42:48.826597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:48.826597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:48.826597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:48.826597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:48.826597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:48.826597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:48.827318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:48.827318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:48.827318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:48.827318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:48.827318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:48.827318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:48.827318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:48.827318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:48.827318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:48.827318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:48.827318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:48.827318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:48.827318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:48.828189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:48.828189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:48.828189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:48.828189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:48.828189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:48.828189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:48.828189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:48.828189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:48.828189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:48.828189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:48.828189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:48.828189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:48.828189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:48.828189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:48.828189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:48.829015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:48.829015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:48.829015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:48.829015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:42:48.829015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:48.829015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:48.829015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:48.829015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:48.829015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:48.829015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:48.829015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:48.829015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:48.829015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:48.829015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:48.829015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:48.829015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:48.829015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:48.829015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:48.829015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:48.831333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:48.831333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:48.831333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:48.831333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:48.831333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:48.831333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:49.001544 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:49.001942 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:49.002143 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:49.003264 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:49.003631 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:49.003906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.004768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:49.004768 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:49.004768 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:49.004768 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:49.004768 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:49.004768 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:49.004768 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.004768 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:49.006333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.006333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:49.006333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:49.006333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:49.006333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:49.006333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:42:49.006333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:49.006333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:49.006333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:49.006333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:49.006333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:49.006333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:49.007211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:49.007211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:49.007211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:49.007211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:49.007211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:49.007211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:49.007211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:49.007211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:49.007211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:49.007211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:49.007211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:49.007211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:49.007211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:49.007953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.007953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:49.008071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.008071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:49.008071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:49.008071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:49.008071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:49.008071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:49.008071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:49.008071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:49.008071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:49.008071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:49.008071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:49.008071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:49.008071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:49.008071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:49.008071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:49.008071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:49.008950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:42:49.008950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:49.008950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:49.008950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:49.008950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:49.008950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:49.008950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:49.008950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:49.008950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:49.008950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:49.008950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:49.008950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:49.008950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:49.008950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:49.008950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:49.008950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:49.008950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:49.008950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:49.008950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:49.009878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:49.009878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.009878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:49.026425 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:49.026793 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:49.026989 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:49.027125 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:42:49.027695 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:49.027995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.028781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:49.028781 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:49.028781 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:49.028781 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:49.028781 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:49.028781 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:49.028781 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.028781 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:49.029965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.029965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:42:49.029965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:49.029965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:49.029965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:49.029965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:49.029965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:49.029965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:49.029965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:49.029965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:49.029965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:49.029965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:42:49.029965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:49.029965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:42:49.029965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:42:49.030726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:49.030726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:42:49.030726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:49.030726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:42:49.030726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:42:49.030726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:49.030726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:42:49.030726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.030726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:42:49.031256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.031256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:49.031256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:49.031256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:49.031256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:49.031256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:42:49.031256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:42:49.031256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:42:49.031256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:49.031256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:49.031256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:49.031256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:49.031901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:49.031901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:49.031901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:49.031901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:49.031901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:49.031901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:49.031901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:49.031901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:49.031901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:49.031901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:49.031901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:49.031901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:49.031901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:49.031901 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:49.032483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:49.032483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:49.032483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.032483 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:49.032906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.032906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:49.032906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:49.032906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:49.032906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:49.032906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:49.032906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:49.032906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:49.032906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:49.032906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:49.032906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:49.032906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:49.032906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:49.032906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:49.032906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:49.032906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:49.033644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:42:49.033644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:42:49.033644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:42:49.033644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:49.033644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:49.033644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:49.033644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:49.033644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:49.033644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:49.033644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:49.033644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:49.033644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:49.033644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:49.033644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:49.033644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:49.033644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:49.033644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:49.033644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:49.034489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:49.034489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:49.034489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:49.034489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:49.034489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:49.034489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.034489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:49.047187 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:49.047430 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:49.047642 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:49.047871 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:49.048287 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:49.048521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.049142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:49.049142 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:49.049142 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:49.049142 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:49.049142 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:49.049142 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:49.049142 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.049142 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:49.050281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.050281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:42:49.050281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:49.050281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:49.050281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:49.050281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:49.050281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:49.050281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:49.050281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:49.050281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:49.050281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:49.050281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:42:49.050281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:49.050281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:42:49.050281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:42:49.050944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:49.050944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:42:49.050944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:49.050944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:42:49.050944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:42:49.050944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:49.050944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:42:49.050944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.050944 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:42:49.051665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.051665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:49.051665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:49.051665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:49.051665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:49.051665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:42:49.051665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:42:49.051665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:49.051665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:49.051665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:49.051665 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:49.052019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:49.052019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:49.052019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:49.052019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:49.052019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:49.052019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:49.052019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:49.052019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:49.052019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:49.052019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:49.052019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:49.052019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:49.052019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:49.052019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:49.052383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:49.052383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:49.052383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.052383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:49.053464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.053464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:49.053464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:49.053464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:49.053464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:49.053464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:49.053464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:49.053464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:49.053464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:49.053464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:49.053464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:49.053464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:49.053464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:49.053464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:49.053464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:49.053464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:49.053464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:42:49.053999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:42:49.053999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:49.053999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:49.053999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:49.053999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:49.053999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:49.053999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:49.053999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:49.053999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:49.053999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:49.053999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:49.053999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:49.053999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:49.053999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:49.053999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:49.053999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:49.053999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:49.053999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:49.053999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:49.054515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:49.054515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:49.054515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.054515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:49.121022 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:49.121363 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:49.121532 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:49.121809 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:49.122304 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:49.122519 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:49.122630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.123195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:49.123195 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:49.123195 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:49.123195 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:49.123195 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:49.123195 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:49.123195 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.123195 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:49.127110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.127110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:49.127110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:49.127110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:49.127110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:49.127110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:42:49.127110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:49.127110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:49.127110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:49.127110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:49.127110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:49.127110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:49.127975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:49.127975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:49.127975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:49.127975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:49.127975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:49.127975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:49.127975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:49.127975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:49.127975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:49.127975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:49.127975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:49.127975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:49.127975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:49.128328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.128328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:49.128762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.128762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:49.128762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:49.128762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:49.128762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:49.128762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:49.128762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:49.128762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:49.128762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:49.128762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:49.128762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:49.128762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:49.128762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:49.128762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:49.128762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:49.128762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:49.129205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:42:49.129205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:49.129205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:49.129205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:49.129205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:49.129205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:49.129205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:49.129205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:49.129205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:49.129205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:49.129205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:49.129205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:49.129205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:49.129205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:49.129205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:49.129205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:49.129205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:49.129205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:49.129205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:49.129716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:49.129716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:49.129716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:53.787853 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:53.788624 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:53.788907 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:53.790247 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:53.825178 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:53.825625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:53.826537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:53.826537 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:53.826537 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:53.826537 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:53.826537 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:53.826537 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:53.826537 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:53.826537 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:53.827555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:53.827555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:53.827555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:53.827555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:53.827555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:53.827555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:42:53.827555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:53.827555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:53.827555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:53.827555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:53.827555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:53.827555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:53.828205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:53.828205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:53.828205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:53.828205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:53.828205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:53.828205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:53.828205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:53.828205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:53.828205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:53.828205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:53.828205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:53.828205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:53.828205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:53.828811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:53.828811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:53.828811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:53.828811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:53.828811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:53.828811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:53.828811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:53.828811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:53.828811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:53.828811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:53.828811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:53.828811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:53.828811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:53.828811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:53.828811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:53.829447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:53.829447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:53.829447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:53.829447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:42:53.829447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:53.829447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:53.829447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:53.829447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:53.829447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:53.829447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:53.829447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:53.829447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:53.829447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:53.829447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:53.829447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:53.829447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:53.829447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:53.829447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:53.829447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:53.830284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:53.830284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:53.830284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:53.830284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:53.830284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:53.830284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:54.002165 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:54.002842 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:54.003216 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:54.004398 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:54.004819 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:54.005138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.005964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:54.005964 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:54.005964 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:54.005964 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:54.005964 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:54.005964 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:54.005964 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.005964 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:54.007623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.007623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:54.007623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:54.007623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:54.007623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:54.007623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:42:54.007623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:54.007623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:54.007623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:54.007623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:54.007623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:54.007623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:54.008193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:54.008193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:54.008193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:54.008193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:54.008193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:54.008193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:54.008193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:54.008193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:54.008193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:54.008193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:54.008193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:54.008193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:54.008193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:54.008789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.008789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:54.009480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.009480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:54.009480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:54.009480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:54.009480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:54.009480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:54.009480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:54.009480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:54.009480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:54.009480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:54.009480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:54.009480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:54.009480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:54.009480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:54.009480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:54.009480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:54.010263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:42:54.010263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:54.010263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:54.010263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:54.010263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:54.010263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:54.010263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:54.010263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:54.010263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:54.010263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:54.010263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:54.010263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:54.010263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:54.010263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:54.010263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:54.010263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:54.010263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:54.010263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:54.010263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:54.011010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:54.011010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.011010 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:54.026999 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:54.027469 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:54.027659 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:54.027885 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:42:54.028497 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:54.028840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.029612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:54.029612 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:54.029612 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:54.029612 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:54.029612 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:54.029612 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:54.029612 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.029612 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:54.030971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.030971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:42:54.030971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:54.030971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:54.030971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:54.030971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:54.030971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:54.030971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:54.030971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:54.030971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:54.030971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:54.030971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:42:54.030971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:54.030971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:42:54.030971 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:42:54.031645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:54.031645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:42:54.031645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:54.031645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:42:54.031645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:42:54.031645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:54.031645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:42:54.031645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.031645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:42:54.032267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.032267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:54.032267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:54.032267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:54.032267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:54.032267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:42:54.032267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:42:54.032267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:42:54.032267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:54.032267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:54.032267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:54.032267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:54.032814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:54.032814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:54.032814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:54.032814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:54.032814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:54.032814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:54.032814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:54.032814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:54.032814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:54.032814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:54.032814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:54.032814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:54.032814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:54.032814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:54.033481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:54.033481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:54.033481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.033481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:54.033986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.033986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:54.033986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:54.033986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:54.033986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:54.033986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:54.033986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:54.033986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:54.033986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:54.033986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:54.033986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:54.033986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:54.033986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:54.033986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:54.033986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:54.033986 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:54.034705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:42:54.034705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:42:54.034705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:42:54.034705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:54.034705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:54.034705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:54.034705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:54.034705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:54.034705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:54.034705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:54.034705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:54.034705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:54.034705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:54.034705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:54.034705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:54.034705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:54.034705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:54.034705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:54.036151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:54.036151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:54.036151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:54.036151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:54.036151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:54.036151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.036151 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:54.047621 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:54.047999 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:54.048185 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:54.048410 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:54.048981 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:54.049324 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.050103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:54.050103 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:54.050103 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:54.050103 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:54.050103 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:54.050103 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:54.050103 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.050103 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:54.051404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.051404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:42:54.051404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:54.051404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:54.051404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:54.051404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:54.051404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:54.051404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:54.051404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:54.051404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:54.051404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:54.051404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:42:54.051404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:54.051404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:42:54.051404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:42:54.052105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:54.052105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:42:54.052105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:54.052105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:42:54.052105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:42:54.052105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:54.052105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:42:54.052105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.052105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:42:54.053011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.053011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:54.053011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:54.053011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:54.053011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:54.053011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:42:54.053011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:42:54.053011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:54.053011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:54.053011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:54.053011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:54.053489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:54.053489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:54.053489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:54.053489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:54.053489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:54.053489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:54.053489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:54.053489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:54.053489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:54.053489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:54.053489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:54.053489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:54.053489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:54.053489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:54.054337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:54.054337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:54.054337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.054337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:54.054904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.054904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:54.054904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:54.054904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:54.054904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:54.054904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:54.054904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:54.054904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:54.054904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:54.054904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:54.054904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:54.054904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:54.054904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:54.054904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:54.054904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:54.054904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:54.054904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:42:54.055395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:42:54.055395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:54.055395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:54.055395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:54.055395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:54.055395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:54.055395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:54.055395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:54.055395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:54.055395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:54.055395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:54.055395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:54.055395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:54.055395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:54.055395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:54.055395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:54.055395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:54.055395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:54.055395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:54.055985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:54.055985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:54.055985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.055985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:54.121573 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:54.122041 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:54.122264 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:54.122511 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:54.123222 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:54.123630 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:54.123936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.124812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:54.124812 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:54.124812 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:54.124812 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:54.124812 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:54.124812 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:54.124812 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.124812 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:54.126252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.126252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:54.126252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:54.126252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:54.126252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:54.126252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:42:54.126252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:54.126252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:54.126252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:54.126252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:54.126252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:54.126252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:54.126820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:54.126820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:54.126820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:54.126820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:54.126820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:54.126820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:54.126820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:54.126820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:54.126820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:54.126820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:54.126820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:54.126820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:54.126820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:54.127453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.127453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:54.127453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.127453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:54.127453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:54.127453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:54.127453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:54.127453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:54.127453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:54.127453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:54.127453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:54.127453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:54.127453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:54.127453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:54.127453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:54.128071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:54.128071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:54.128071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:54.128071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:42:54.128071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:54.128071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:54.128071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:54.128071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:54.128071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:54.128071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:54.128071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:54.128071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:54.128071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:54.128071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:54.128071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:54.128071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:54.128071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:54.128071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:54.128071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:54.128906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:54.128906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:54.128906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:54.128906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:54.128906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:54.128906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:58.788628 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:58.789372 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:58.789462 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:58.790617 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:58.829229 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:58.829618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:58.830453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:58.830453 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:58.830453 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:58.830453 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:58.830453 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:58.830453 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:58.830453 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:58.830453 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:58.831210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:58.831210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:58.831210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:58.831210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:58.831210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:58.831210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:42:58.831210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:58.831210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:58.831210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:58.831210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:58.831210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:58.831210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:58.832162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:58.832162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:58.832162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:58.832162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:58.832162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:58.832162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:58.832162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:58.832162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:58.832162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:58.832162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:58.832162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:58.832162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:58.832162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:58.833174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:58.833174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:58.833174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:58.833174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:58.833174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:58.833174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:58.833174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:58.833174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:58.833174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:58.833174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:58.833174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:58.833174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:58.833174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:58.833174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:58.833174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:58.835226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:58.835226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:58.835226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:58.835226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:42:58.835226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:58.835226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:58.835226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:58.835226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:58.835226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:58.835226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:58.835226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:58.835226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:58.835226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:58.835226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:58.835226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:58.835226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:58.835226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:58.835226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:58.835226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:58.836950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:58.836950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:58.836950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:58.836950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:58.836950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:58.836950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:59.003023 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:59.003545 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:59.003787 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:59.005983 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:59.007028 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:59.007412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.008747 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:59.008747 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:59.008747 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:59.008747 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:59.008747 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:59.008747 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:59.008747 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.008747 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:59.010339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.005 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.010339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:59.010339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:59.010339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:59.010339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:59.010339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:42:59.010339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:59.010339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:59.010339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:59.010339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:59.010339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:59.010339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:59.011068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:59.011068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:59.011068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:59.011068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:59.011068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:59.011068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:59.011068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:59.011068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:59.011068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:59.011068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:59.011068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:59.011068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:59.011068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:59.011719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.011719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:59.011985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.011985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:59.011985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:59.011985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:59.011985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:59.011985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:59.011985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:59.011985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:59.011985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:59.011985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:59.011985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:59.011985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:59.011985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:59.011985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:59.011985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:59.011985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:59.012881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:42:59.012881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:59.012881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:59.012881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:59.012881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:59.012881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:59.012881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:59.012881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:59.012881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:59.012881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:59.012881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:59.012881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:59.012881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:59.012881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:59.012881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:59.012881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:59.012881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:59.012881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:59.012881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:59.013754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:59.013754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.013754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:59.027518 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:59.027902 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:59.028037 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:59.028225 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:42:59.028757 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:59.029053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.029788 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:59.029788 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:59.029788 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:59.029788 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:59.029788 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:59.029788 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:59.029788 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.029788 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:59.030914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.030914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:42:59.030914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:59.030914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:59.030914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:59.030914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:59.030914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:59.030914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:59.030914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:59.030914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:59.030914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:59.030914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:42:59.030914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:59.030914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:42:59.030914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:42:59.031629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:59.031629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:42:59.031629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:59.031629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:42:59.031629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:42:59.031629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:59.031629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:42:59.031629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.031629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:42:59.032195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.032195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:59.032195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:59.032195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:59.032195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:59.032195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:42:59.032195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:42:59.032195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:42:59.032195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:59.032195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:59.032195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:59.032195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:59.032979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:59.032979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:59.032979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:59.032979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:59.032979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:59.032979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:59.032979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:59.032979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:59.032979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:59.032979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:59.032979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:59.032979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:59.032979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:59.032979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:59.033750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:59.033750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:59.033750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.033750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:59.033750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.033750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:59.033750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:59.033750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:59.033750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:59.033750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:59.033750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:59.033750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:59.033750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:59.033750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:59.034354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:59.034354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:59.034354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:59.034354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:59.034354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:59.034354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:59.034354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:42:59.034354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:42:59.034354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:42:59.034354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:59.034354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:59.034354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:59.034354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:59.034354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:59.034354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:59.034354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:59.034354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:59.034354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:59.034354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:59.035185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:59.035185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:59.035185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:59.035185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:59.035185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:59.035185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:59.035185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:59.035185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:59.035185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:59.035185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:59.035185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.035185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:59.048027 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:59.048277 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:59.048385 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:59.048504 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:59.048785 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:59.048993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.049601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:59.049601 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:59.049601 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:59.049601 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:59.049601 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:59.049601 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:59.049601 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.049601 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:59.050699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.050699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:42:59.050699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:59.050699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:59.050699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:59.050699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:59.050699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:59.050699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:59.050699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:59.050699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:42:59.050699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:59.050699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:42:59.050699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:59.050699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:42:59.050699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:42:59.051119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:59.051119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:42:59.051119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:59.051119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:42:59.051119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:42:59.051119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:59.051119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:42:59.051119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.051119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:42:59.052042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.052042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:59.052042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:59.052042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:59.052042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:59.052042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:42:59.052042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:42:59.052042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:59.052042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:59.052042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:59.052042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:59.052484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:59.052484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:59.052484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:59.052484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:59.052484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:59.052484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:59.052484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:59.052484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:59.052484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:59.052484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:59.052484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:59.052484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:59.052484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:59.052484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:59.052966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:59.052966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:59.052966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.052966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:59.053690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.053690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:59.053690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:59.053690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:59.053690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:59.053690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:59.053690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:59.053690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:59.053690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:59.053690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:59.053690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:59.053690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:59.053690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:59.053690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:59.053690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:59.053690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:59.053690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:42:59.054220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:42:59.054220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:42:59.054220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:42:59.054220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:59.054220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:59.054220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:59.054220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:59.054220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:59.054220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:59.054220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:59.054220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:59.054220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:59.054220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:59.054220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:59.054220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:59.054220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:59.054220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:59.054220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:59.054220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:59.054846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:59.054846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:59.054846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.054846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:42:59.122104 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:42:59.122522 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:42:59.122714 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:42:59.122986 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:42:59.123535 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:59.123840 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:42:59.123979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.124805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:42:59.124805 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:59.124805 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:42:59.124805 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:42:59.124805 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:59.124805 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:42:59.124805 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.124805 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:42:59.126164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.126164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:42:59.126164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:59.126164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:42:59.126164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:59.126164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:42:59.126164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:59.126164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:59.126164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:59.126164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:59.126164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:59.126164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:59.126866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:42:59.126866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:59.126866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:42:59.126866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:59.126866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:42:59.126866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:42:59.126866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:59.126866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:42:59.126866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:59.126866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:42:59.126866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:42:59.126866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:59.126866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:42:59.127333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.127333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:42:59.127668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.127668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:42:59.127668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:42:59.127668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:42:59.127668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:42:59.127668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:42:59.127668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:59.127668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:42:59.127668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:42:59.127668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:59.127668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:42:59.127668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:42:59.127668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:42:59.127668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:42:59.127668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:42:59.127668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:42:59.128148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:42:59.128148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:42:59.128148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:59.128148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:59.128148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:42:59.128148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:42:59.128148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:42:59.128148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:42:59.128148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:42:59.128148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:42:59.128148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:42:59.128148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:42:59.128148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:42:59.128148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:42:59.128148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:42:59.128148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:42:59.128148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:42:59.128148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:42:59.128148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:42:59.128642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:42:59.128642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:42:59.128642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:03.789291 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:03.790013 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:03.790088 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:03.791429 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:03.823462 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:03.823621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:03.824039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:03.824039 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:03.824039 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:03.824039 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:03.824039 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:03.824039 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:03.824039 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:03.824039 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:03.824690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:03.824690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:03.824690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:03.824690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:03.824690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:03.824690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:43:03.824690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:03.824690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:03.824690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:03.824690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:03.824690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:03.824690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:03.825179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:03.825179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:03.825179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:03.825179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:03.825179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:03.825179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:03.825179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:03.825179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:03.825179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:03.825179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:03.825179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:03.825179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:03.825179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:03.825640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:03.825640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:03.825640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:03.825640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:03.825640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:03.825640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:03.825640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:03.825640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:03.825640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:03.825640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:03.825640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:03.825640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:03.825640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:03.825640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:03.825640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:03.826120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:03.826120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:03.826120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:03.826120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:43:03.826120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:03.826120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:03.826120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:03.826120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:03.826120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:03.826120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:03.826120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:03.826120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:03.826120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:03.826120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:03.826120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:03.826120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:03.826120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:03.826120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:03.826120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:03.826840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:03.826840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:03.826840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:03.826840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:03.826840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:03.826840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:04.003596 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:04.004153 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:04.004287 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:04.005261 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:04.005536 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:04.005805 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.006634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:04.006634 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:04.006634 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:04.006634 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:04.006634 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:04.006634 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:04.006634 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.006634 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:04.008321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.008321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:04.008321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:04.008321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:04.008321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:04.008321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:43:04.008321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:04.008321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:04.008321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:04.008321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:04.008321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:04.008321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:04.008877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:04.008877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:04.008877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:04.008877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:04.008877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:04.008877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:04.008877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:04.008877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:04.008877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:04.008877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:04.008877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:04.008877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:04.008877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:04.009409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.009409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:04.010296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.010296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:04.010296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:04.010296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:04.010296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:04.010296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:04.010296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:04.010296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:04.010296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:04.010296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:04.010296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:04.010296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:04.010296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:04.010296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:04.010296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:04.010296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:04.010979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:43:04.010979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:04.010979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:04.010979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:04.010979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:04.010979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:04.010979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:04.010979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:04.010979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:04.010979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:04.010979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:04.010979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:04.010979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:04.010979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:04.010979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:04.010979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:04.010979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:04.010979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:04.010979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:04.011813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:04.011813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.011813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:04.027941 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:04.028212 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:04.028342 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:04.028439 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:43:04.028785 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:04.029009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.029462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:04.029462 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:04.029462 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:04.029462 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:04.029462 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:04.029462 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:04.029462 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.029462 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:04.030218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.030218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:43:04.030218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:04.030218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:04.030218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:04.030218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:04.030218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:04.030218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:04.030218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:04.030218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:04.030218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:04.030218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:43:04.030218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:04.030218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:43:04.030218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:43:04.030909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:04.030909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:43:04.030909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:04.030909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:43:04.030909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:43:04.030909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:04.030909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:43:04.030909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.030909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:43:04.031282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.031282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:04.031282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:04.031282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:04.031282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:04.031282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:43:04.031282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:43:04.031282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:43:04.031282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:04.031282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:04.031282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:04.031282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:04.031923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:04.031923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:04.031923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:04.031923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:04.031923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:04.031923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:04.031923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:04.031923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:04.031923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:04.031923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:04.031923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:04.031923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:04.031923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:04.031923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:04.032540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:04.032540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:04.032540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.032540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:04.032540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.032540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:04.032540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:04.032540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:04.032540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:04.032540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:04.032540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:04.032540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:04.032540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:04.032540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:04.033156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:04.033156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:04.033156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:04.033156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:04.033156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:04.033156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:04.033156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:43:04.033156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:43:04.033156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:43:04.033156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:04.033156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:04.033156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:04.033156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:04.033156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:04.033156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:04.033156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:04.033156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:04.033156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:04.033156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:04.033995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:04.033995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:04.033995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:04.033995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:04.033995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:04.033995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:04.033995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:04.033995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:04.033995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:04.033995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:04.033995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.033995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:04.048415 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:04.048711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:04.048861 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:04.049033 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:04.049707 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:04.050209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.051092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:04.051092 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:04.051092 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:04.051092 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:04.051092 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:04.051092 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:04.051092 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.051092 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:04.052112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.052112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:43:04.052112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:04.052112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:04.052112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:04.052112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:04.052112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:04.052112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:04.052112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:04.052112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:04.052112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:04.052112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:43:04.052112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:04.052112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:43:04.052112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:43:04.052746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:04.052746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:43:04.052746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:04.052746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:43:04.052746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:43:04.052746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:04.052746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:43:04.052746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.052746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:43:04.053132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.053132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:04.053132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:04.053132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:04.053132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:04.053132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:43:04.053132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:43:04.053132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:04.053132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:04.053132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:04.053132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:04.053616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:04.053616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:04.053616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:04.053616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:04.053616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:04.053616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:04.053616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:04.053616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:04.053616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:04.053616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:04.053616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:04.053616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:04.053616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:04.053616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:04.054216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:04.054216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:04.054216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.054216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:04.054216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.054216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:04.054216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:04.054216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:04.054216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:04.054216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:04.054216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:04.054216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:04.054216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:04.054216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:04.054841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:04.054841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:04.054841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:04.054841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:04.054841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:04.054841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:04.054841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:43:04.054841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:43:04.054841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:04.054841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:04.054841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:04.054841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:04.054841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:04.054841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:04.054841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:04.054841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:04.054841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:04.054841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:04.054841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:04.055987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:04.055987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:04.055987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:04.055987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:04.055987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:04.055987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:04.055987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:04.055987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:04.055987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:04.055987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.055987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:04.122585 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:04.123022 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:04.123200 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:04.123514 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:04.124070 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:04.124463 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:04.124763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.125544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:04.125544 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:04.125544 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:04.125544 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:04.125544 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:04.125544 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:04.125544 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.125544 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:04.127007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.127007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:04.127007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:04.127007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:04.127007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:04.127007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:43:04.127007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:04.127007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:04.127007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:04.127007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:04.127007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:04.127007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:04.127540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:04.127540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:04.127540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:04.127540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:04.127540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:04.127540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:04.127540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:04.127540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:04.127540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:04.127540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:04.127540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:04.127540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:04.127540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:04.128128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.128128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:04.128836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.128836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:04.128836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:04.128836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:04.128836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:04.128836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:04.128836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:04.128836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:04.128836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:04.128836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:04.128836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:04.128836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:04.128836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:04.128836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:04.128836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:04.128836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:04.129510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:43:04.129510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:04.129510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:04.129510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:04.129510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:04.129510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:04.129510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:04.129510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:04.129510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:04.129510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:04.129510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:04.129510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:04.129510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:04.129510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:04.129510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:04.129510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:04.129510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:04.129510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:04.129510 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:04.130502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:04.130502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:04.130502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:08.790249 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:08.791021 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:08.791021 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:08.792083 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:08.793665 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=240 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:43:08.835385 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:08.835714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:08.836618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:08.836618 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:08.836618 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:08.836618 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:08.836618 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:08.836618 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:08.836618 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:08.836618 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:08.837617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.046 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:08.837617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:08.837617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:08.837617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:08.837617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:08.837617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:43:08.837617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:08.837617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:08.837617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:08.837617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:08.837617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:08.837617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:08.838612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:08.838612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:08.838612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:08.838612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:08.838612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:08.838612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:08.838612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:08.838612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:08.838612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:08.838612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:08.838612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:08.838612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:08.838612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:08.839457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:08.839457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:08.839457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:08.839457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:08.839457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:08.839457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:08.839457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:08.839457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:08.839457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:08.839457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:08.839457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:08.839457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:08.839457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:08.839457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:08.839457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:08.840308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:08.840308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:08.840308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:08.840308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:43:08.840308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:08.840308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:08.840308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:08.840308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:08.840308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:08.840308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:08.840308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:08.840308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:08.840308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:08.840308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:08.840308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:08.840308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:08.840308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:08.840308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:08.840308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:08.841298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:08.841298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:08.841298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:08.841298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:08.841298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:08.841298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:09.004332 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:09.004864 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:09.004994 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:09.005848 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:09.006144 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:09.006144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.006918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:09.006918 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:09.006918 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:09.006918 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:09.006918 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:09.006918 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:09.006918 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.006918 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:09.008474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.008474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:09.008474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:09.008474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:09.008474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:09.008474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:43:09.008474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:09.008474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:09.008474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:09.008474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:09.008474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:09.008474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:09.009115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:09.009115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:09.009115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:09.009115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:09.009115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:09.009115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:09.009115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:09.009115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:09.009115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:09.009115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:09.009115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:09.009115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:09.009115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:09.009751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.009751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:09.010304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.010304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:09.010304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:09.010304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:09.010304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:09.010304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:09.010304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:09.010304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:09.010304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:09.010304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:09.010304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:09.010304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:09.010304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:09.010304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:09.010304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:09.010304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:09.010846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:43:09.010846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:09.010846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:09.010846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:09.010846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:09.010846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:09.010846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:09.010846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:09.010846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:09.010846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:09.010846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:09.010846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:09.010846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:09.010846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:09.010846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:09.010846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:09.010846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:09.010846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:09.010846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:09.011354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:09.011354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.011354 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:09.028380 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:09.028873 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:09.029075 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:09.029286 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:43:09.029878 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:09.030215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.031089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:09.031089 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:09.031089 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:09.031089 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:09.031089 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:09.031089 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:09.031089 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.031089 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:09.032339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.032339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:43:09.032339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:09.032339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:09.032339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:09.032339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:09.032339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:09.032339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:09.032339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:09.032339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:09.032339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:09.032339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:43:09.032339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:09.032339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:43:09.032339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:43:09.033281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:09.033281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:43:09.033281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:09.033281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:43:09.033281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:43:09.033281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:09.033281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:43:09.033281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.033281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:43:09.034229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.034229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:09.034229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:09.034229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:09.034229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:09.034229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:43:09.034229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:43:09.034229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:43:09.034229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:09.034229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:09.034229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:09.034229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:09.035087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:09.035087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:09.035087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:09.035087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:09.035087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:09.035087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:09.035087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:09.035087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:09.035087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:09.035087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:09.035087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:09.035087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:09.035087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:09.035087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:09.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:09.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:09.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:09.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:09.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:09.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:09.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:09.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:09.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:09.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:09.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:09.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:09.036634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:09.036634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:09.036634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:09.036634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:09.036634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:09.036634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:09.036634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:43:09.036634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:43:09.036634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:43:09.036634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:09.036634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:09.036634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:09.036634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:09.036634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:09.036634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:09.036634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:09.036634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:09.036634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:09.036634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:09.037739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:09.037739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:09.037739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:09.037739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:09.037739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:09.037739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:09.037739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:09.037739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:09.037739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:09.037739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:09.037739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.037739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:09.048935 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:09.049333 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:09.049521 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:09.049764 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:09.050203 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:09.050462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.051113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:09.051113 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:09.051113 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:09.051113 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:09.051113 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:09.051113 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:09.051113 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.051113 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:09.052779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.052779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:43:09.052779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:09.052779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:09.052779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:09.052779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:09.052779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:09.052779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:09.052779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:09.052779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:09.052779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:09.052779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:43:09.052779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:09.052779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:43:09.052779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:43:09.053462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:09.053462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:43:09.053462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:09.053462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:43:09.053462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:43:09.053462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:09.053462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:43:09.053462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.053462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:43:09.054834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.054834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:09.054834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:09.054834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:09.054834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:09.054834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:43:09.054834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:43:09.054834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:09.054834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:09.054834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:09.054834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:09.055495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:09.055495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:09.055495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:09.055495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:09.055495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:09.055495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:09.055495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:09.055495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:09.055495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:09.055495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:09.055495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:09.055495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:09.055495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:09.055495 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:09.056330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:09.056330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:09.056330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.056330 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:09.056941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.056941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:09.056941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:09.056941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:09.056941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:09.056941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:09.056941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:09.056941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:09.056941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:09.056941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:09.056941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:09.056941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:09.056941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:09.056941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:09.056941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:09.056941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:09.056941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:43:09.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:43:09.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:09.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:09.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:09.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:09.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:09.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:09.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:09.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:09.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:09.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:09.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:09.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:09.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:09.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:09.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:09.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:09.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:09.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:09.058941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:09.058941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:09.058941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.058941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:09.123373 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:09.123977 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:09.124265 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:09.124644 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:09.125297 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:09.125696 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:09.125972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.126963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:09.126963 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:09.126963 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:09.126963 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:09.126963 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:09.126963 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:09.126963 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.126963 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:09.128716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.128716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:09.128716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:09.128716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:09.128716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:09.128716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:43:09.128716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:09.128716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:09.128716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:09.128716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:09.128716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:09.128716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:09.129872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:09.129872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:09.129872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:09.129872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:09.129872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:09.129872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:09.129872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:09.129872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:09.129872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:09.129872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:09.129872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:09.129872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:09.129872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:09.130755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.130755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:09.131031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.131031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:09.131031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:09.131031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:09.131031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:09.131031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:09.131031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:09.131031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:09.131031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:09.131031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:09.131031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:09.131031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:09.131031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:09.131031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:09.131031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:09.131031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:09.131500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:43:09.131500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:09.131500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:09.131500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:09.131500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:09.131500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:09.131500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:09.131500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:09.131500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:09.131500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:09.131500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:09.131500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:09.131500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:09.131500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:09.131500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:09.131500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:09.131500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:09.131500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:09.131500 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:09.132115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:09.132115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:09.132115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:13.790996 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:13.791783 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:13.791783 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:13.792819 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:13.829069 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:13.829406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:13.830423 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:13.830423 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:13.830423 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:13.830423 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:13.830423 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:13.830423 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:13.830423 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:13.830423 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:13.832112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:13.832112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:13.832112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:13.832112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:13.832112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:13.832112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:43:13.832112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:13.832112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:13.832112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:13.832112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:13.832112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:13.832112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:13.832763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:13.832763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:13.832763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:13.832763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:13.832763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:13.832763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:13.832763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:13.832763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:13.832763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:13.832763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:13.832763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:13.832763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:13.832763 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:13.833292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:13.833292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:13.834164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:13.834164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:13.834164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:13.834164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:13.834164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:13.834164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:13.834164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:13.834164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:13.834164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:13.834164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:13.834164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:13.834164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:13.834164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:13.834164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:13.834164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:13.834164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:13.834958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:43:13.834958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:13.834958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:13.834958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:13.834958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:13.834958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:13.834958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:13.834958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:13.834958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:13.834958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:13.834958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:13.834958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:13.834958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:13.834958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:13.834958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:13.834958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:13.834958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:13.834958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:13.834958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:13.835823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:13.835823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:13.835823 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:14.005100 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:14.005657 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:14.005805 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:14.007026 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:14.007998 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:14.007998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.008274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:14.008274 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:14.008274 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:14.008274 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:14.008274 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:14.008274 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:14.008274 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.008274 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:14.009939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.009939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:14.009939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:14.009939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:14.009939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:14.009939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:43:14.009939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:14.009939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:14.009939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:14.009939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:14.009939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:14.009939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:14.010279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:14.010279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:14.010279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:14.010279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:14.010279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:14.010279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:14.010279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:14.010279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:14.010279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:14.010279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:14.010279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:14.010279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:14.010279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:14.010593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.010593 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:14.011724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.011724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:14.011724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:14.011724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:14.011724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:14.011724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:14.011724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:14.011724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:14.011724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:14.011724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:14.011724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:14.011724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:14.011724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:14.011724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:14.011724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:14.011724 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:14.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:43:14.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:14.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:14.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:14.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:14.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:14.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:14.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:14.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:14.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:14.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:14.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:14.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:14.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:14.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:14.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:14.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:14.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:14.012251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:14.012824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:14.012824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.012824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:14.028918 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:14.029456 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:14.029700 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:14.030034 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:43:14.030814 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:14.031081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.031970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:14.031970 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:14.031970 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:14.031970 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:14.031970 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:14.031970 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:14.031970 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.031970 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:14.033311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.033311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:43:14.033311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:14.033311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:14.033311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:14.033311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:14.033311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:14.033311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:14.033311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:14.033311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:14.033311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:14.033311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:43:14.033311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:14.033311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:43:14.033311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:43:14.034118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:14.034118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:43:14.034118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:14.034118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:43:14.034118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:43:14.034118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:14.034118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:43:14.034118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.034118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:43:14.034533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.034533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:14.034533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:14.034533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:14.034533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:14.034533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:43:14.034533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:43:14.034533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:43:14.034533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:14.034533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:14.034533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:14.034533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:14.035108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:14.035108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:14.035108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:14.035108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:14.035108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:14.035108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:14.035108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:14.035108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:14.035108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:14.035108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:14.035108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:14.035108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:14.035108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:14.035108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:14.035666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:14.035666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:14.035666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.035666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:14.035666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.035666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:14.035666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:14.035666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:14.035666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:14.035666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:14.035666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:14.035666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:14.035666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:14.035666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:14.036904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:14.036904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:14.036904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:14.036904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:14.036904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:14.036904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:14.036904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:43:14.036904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:43:14.036904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:43:14.036904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:14.036904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:14.036904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:14.036904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:14.036904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:14.036904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:14.036904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:14.036904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:14.036904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:14.036904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:14.037804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:14.037804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:14.037804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:14.037804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:14.037804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:14.037804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:14.037804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:14.037804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:14.037804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:14.037804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:14.037804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.037804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:14.049573 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:14.049728 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:14.049800 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:14.049893 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:14.050435 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:14.050933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.051714 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:14.051714 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:14.051714 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:14.051714 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:14.051714 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:14.051714 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:14.051714 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.051714 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:14.052266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.052266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:43:14.052266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:14.052266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:14.052266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:14.052266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:14.052266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:14.052266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:14.052266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:14.052266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:14.052266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:14.052266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:43:14.052266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:14.052266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:43:14.052266 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:43:14.053026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:14.053026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:43:14.053026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:14.053026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:43:14.053026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:43:14.053026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:14.053026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:43:14.053026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.053026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:43:14.053026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.053026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:14.053026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:14.053026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:14.053026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:14.053660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:43:14.053660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:43:14.053660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:14.053660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:14.053660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:14.053660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:14.053660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:14.053660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:14.053660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:14.053660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:14.053660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:14.053660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:14.053660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:14.054329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:14.054329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:14.054329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:14.054329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:14.054329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:14.054329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:14.054329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:14.054329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:14.054329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:14.054329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.054329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:14.054329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.054992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:14.054992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:14.054992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:14.054992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:14.054992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:14.054992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:14.054992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:14.054992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:14.054992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:14.054992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:14.054992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:14.054992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:14.054992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:14.054992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:14.054992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:14.054992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:43:14.054992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:43:14.054992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:14.054992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:14.056472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:14.056472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:14.056472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:14.056472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:14.056472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:14.056472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:14.056472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:14.056472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:14.056472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:14.056472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:14.056472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:14.056472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:14.056472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:14.056472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:14.056472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:14.056472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:14.056472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:14.056472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:14.056472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.056472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:14.124062 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:14.125039 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:14.125257 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:14.125563 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:14.126282 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:14.126769 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:14.127068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.128007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:14.128007 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:14.128007 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:14.128007 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:14.128007 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:14.128007 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:14.128007 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.128007 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:14.129253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.129253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:14.129253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:14.129253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:14.129253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:14.129253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:43:14.129253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:14.129253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:14.129253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:14.129253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:14.129253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:14.129253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:14.129852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:14.129852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:14.129852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:14.129852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:14.129852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:14.129852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:14.129852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:14.129852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:14.129852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:14.129852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:14.129852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:14.129852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:14.129852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:14.130473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.130473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:14.130473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.130473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:14.130473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:14.130473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:14.130473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:14.130473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:14.130473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:14.130473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:14.130473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:14.130473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:14.130473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:14.130473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:14.130473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:14.131213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:14.131213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:14.131213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:14.131213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:43:14.131213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:14.131213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:14.131213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:14.131213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:14.131213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:14.131213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:14.131213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:14.131213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:14.131213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:14.131213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:14.131213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:14.131213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:14.131213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:14.131213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:14.131213 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:14.132105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:14.132105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:14.132105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:14.132105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:14.132105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:14.132105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:18.791898 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:18.792975 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:18.792975 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:18.794038 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:18.825865 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:18.826211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:18.827445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:18.827445 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:18.827445 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:18.827445 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:18.827445 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:18.827445 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:18.827445 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:18.827445 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:18.828619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:18.828619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:18.828619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:18.828619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:18.828619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:18.828619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:43:18.828619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:18.828619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:18.828619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:18.828619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:18.828619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:18.828619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:18.829257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:18.829257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:18.829257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:18.829257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:18.829257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:18.829257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:18.829257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:18.829257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:18.829257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:18.829257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:18.829257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:18.829257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:18.829257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:18.829914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:18.829914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:18.830291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:18.830291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:18.830291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:18.830291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:18.830291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:18.830291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:18.830291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:18.830291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:18.830291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:18.830291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:18.830291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:18.830291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:18.830291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:18.830291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:18.830291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:18.830291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:18.830934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:43:18.830934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:18.830934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:18.830934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:18.830934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:18.830934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:18.830934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:18.830934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:18.830934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:18.830934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:18.830934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:18.830934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:18.830934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:18.830934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:18.830934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:18.830934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:18.830934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:18.830934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:18.830934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:18.831586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:18.831586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:18.831586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:19.005972 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:19.006285 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:19.006460 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:19.007402 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:19.007632 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:19.007826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.008950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:19.008950 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:19.008950 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:19.008950 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:19.008950 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:19.008950 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:19.008950 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.008950 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:19.010100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.010100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:19.010100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:19.010100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:19.010100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:19.010100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:43:19.010100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:19.010100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:19.010100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:19.010100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:19.010100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:19.010100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:19.010481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:19.010481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:19.010481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:19.010481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:19.010481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:19.010481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:19.010481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:19.010481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:19.010481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:19.010481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:19.010481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:19.010481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:19.010481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:19.010938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.010938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:19.011752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.011752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:19.011752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:19.011752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:19.011752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:19.011752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:19.011752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:19.011752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:19.011752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:19.011752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:19.011752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:19.011752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:19.011752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:19.011752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:19.011752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:19.011752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:19.012225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:43:19.012225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:19.012225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:19.012225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:19.012225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:19.012225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:19.012225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:19.012225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:19.012225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:19.012225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:19.012225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:19.012225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:19.012225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:19.012225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:19.012225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:19.012225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:19.012225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:19.012225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:19.012225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:19.012776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:19.012776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.012776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:19.029440 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:19.029804 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:19.029980 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:19.030158 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:43:19.030808 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:19.030808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.031350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:19.031350 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:19.031350 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:19.031350 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:19.031350 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:19.031350 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:19.031350 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.031350 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:19.032542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.032542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:43:19.032542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:19.032542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:19.032542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:19.032542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:19.032542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:19.032542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:19.032542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:19.032542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:19.032542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:19.032542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:43:19.032542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:19.032542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:43:19.032542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:43:19.033044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:19.033044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:43:19.033044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:19.033044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:43:19.033044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:43:19.033044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:19.033044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:43:19.033044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.033044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:43:19.033866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.033866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:19.033866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:19.033866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:19.033866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:19.033866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:43:19.033866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:43:19.033866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:43:19.033866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:19.033866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:19.033866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:19.033866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:19.034234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:19.034234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:19.034234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:19.034234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:19.034234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:19.034234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:19.034234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:19.034234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:19.034234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:19.034234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:19.034234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:19.034234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:19.034234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:19.034234 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:19.034614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:19.034614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:19.034614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.034614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:19.035512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.035512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:19.035512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:19.035512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:19.035512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:19.035512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:19.035512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:19.035512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:19.035512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:19.035512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:19.035512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:19.035512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:19.035512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:19.035512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:19.035512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:19.035512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:19.036121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:43:19.036121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:43:19.036121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:43:19.036121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:19.036121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:19.036121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:19.036121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:19.036121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:19.036121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:19.036121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:19.036121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:19.036121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:19.036121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:19.036121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:19.036121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:19.036121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:19.036121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:19.036121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:19.037311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:19.037311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:19.037311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:19.037311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:19.037311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:19.037311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.037311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:19.049835 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:19.050118 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:19.050284 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:19.050472 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:19.050817 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:19.051049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.051617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:19.051617 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:19.051617 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:19.051617 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:19.051617 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:19.051617 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:19.051617 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.051617 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:19.052828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.052828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:43:19.052828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:19.052828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:19.052828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:19.052828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:19.052828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:19.052828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:19.052828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:19.052828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:19.052828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:19.052828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:43:19.052828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:19.052828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:43:19.052828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:43:19.053272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:19.053272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:43:19.053272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:19.053272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:43:19.053272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:43:19.053272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:19.053272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:43:19.053272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.053272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:43:19.054245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.054245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:19.054245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:19.054245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:19.054245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:19.054245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:43:19.054245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:43:19.054245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:19.054245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:19.054245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:19.054245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:19.054913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:19.054913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:19.054913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:19.054913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:19.054913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:19.054913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:19.054913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:19.054913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:19.054913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:19.054913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:19.054913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:19.054913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:19.054913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:19.054913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:19.055584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:19.055584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:19.055584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.055584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:19.056069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.056069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:19.056069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:19.056069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:19.056069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:19.056069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:19.056069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:19.056069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:19.056069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:19.056069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:19.056069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:19.056069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:19.056069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:19.056069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:19.056069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:19.056069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:19.056069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:43:19.057236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:43:19.057236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:19.057236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:19.057236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:19.057236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:19.057236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:19.057236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:19.057236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:19.057236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:19.057236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:19.057236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:19.057236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:19.057236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:19.057236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:19.057236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:19.057236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:19.057236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:19.057236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:19.057236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:19.058122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:19.058122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:19.058122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.058122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:19.125016 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:19.125441 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:19.125665 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:19.126005 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:19.126528 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:19.126846 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:19.127058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.127661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:19.127661 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:19.127661 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:19.127661 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:19.127661 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:19.127661 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:19.127661 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.127661 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:19.129497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.129497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:19.129497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:19.129497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:19.129497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:19.129497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:43:19.129497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:19.129497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:19.129497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:19.129497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:19.129497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:19.129497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:19.129884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:19.129884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:19.129884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:19.129884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:19.129884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:19.129884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:19.129884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:19.129884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:19.129884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:19.129884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:19.129884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:19.129884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:19.129884 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:19.130362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.130362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:19.131774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.131774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:19.131774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:19.131774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:19.131774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:19.131774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:19.131774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:19.131774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:19.131774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:19.131774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:19.131774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:19.131774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:19.131774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:19.131774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:19.131774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:19.131774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:19.132419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:43:19.132419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:19.132419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:19.132419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:19.132419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:19.132419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:19.132419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:19.132419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:19.132419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:19.132419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:19.132419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:19.132419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:19.132419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:19.132419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:19.132419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:19.132419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:19.132419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:19.132419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:19.132419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:19.133014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:19.133014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:19.133014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:23.792800 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:23.793583 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:23.793583 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:23.794559 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:23.831706 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:23.832175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:23.833164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:23.833164 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:23.833164 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:23.833164 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:23.833164 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:23.833164 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:23.833164 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:23.833164 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:23.834152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:23.834152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:23.834152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:23.834152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:23.834152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:23.834152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:43:23.834152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:23.834152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:23.834152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:23.834152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:23.834152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:23.834152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:23.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:23.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:23.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:23.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:23.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:23.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:23.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:23.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:23.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:23.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:23.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:23.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:23.835122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:23.835647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:23.835647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:23.835647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:23.835647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:23.835647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:23.835647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:23.835647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:23.835647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:23.835647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:23.835647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:23.835647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:23.835647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:23.835647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:23.835647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:23.835647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:23.837257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:23.837257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:23.837257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:23.837257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:43:23.837257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:23.837257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:23.837257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:23.837257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:23.837257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:23.837257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:23.837257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:23.837257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:23.837257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:23.837257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:23.837257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:23.837257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:23.837257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:23.837257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:23.837257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:23.838697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:23.838697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:23.838697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:23.838697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:23.838697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:23.838697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:24.032766 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:24.032766 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:24.032766 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:24.032766 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:24.032766 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:24.032766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.034041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:24.034041 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:24.034041 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:24.034041 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:24.034041 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:24.034041 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:24.034041 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.034041 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:24.034041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.034041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:24.034041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:24.034041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:24.034041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:24.034041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:43:24.034041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:24.034786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:24.034786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:24.034786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:24.034786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:24.034786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:24.034786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:24.034786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:24.034786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:24.034786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:24.034786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:24.034786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:24.034786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:24.034786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:24.034786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:24.035440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:24.035440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:24.035440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:24.035440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:24.035440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.035440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:24.035440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.035440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:24.035440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:24.035440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:24.035440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:24.035440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:24.035440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:24.035440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:24.035993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:24.035993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:24.035993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:24.035993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:24.035993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:24.035993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:24.035993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:24.035993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:24.035993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:43:24.035993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:24.035993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:24.035993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:24.035993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:24.035993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:24.035993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:24.035993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:24.035993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:24.035993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:24.035993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:24.035993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:24.036642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:24.036642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:24.036642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:24.036642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:24.036642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:24.036642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:24.036642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:24.036642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:24.036642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.036642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:24.036642 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:24.036642 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:24.037260 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:24.037260 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:43:24.037260 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:24.037260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.037260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:24.037260 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:24.037260 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:24.037260 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:24.037260 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:24.037260 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:24.037260 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.037260 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:24.037666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.037666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:43:24.037666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:24.037666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:24.037666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:24.037666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:24.037666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:24.037666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:24.037666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:24.037666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:24.037666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:24.037666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:43:24.037666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:24.037666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:43:24.037666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:43:24.038498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:24.038498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:43:24.038498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:24.038498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:43:24.038498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:43:24.038498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:24.038498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:43:24.038498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.038498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:43:24.038498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.038498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:24.038498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:24.038498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:24.038498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:24.039087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:43:24.039087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:43:24.039087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:43:24.039087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:24.039087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:24.039087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:24.039087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:24.039087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:24.039087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:24.039087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:24.039087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:24.039087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:24.039087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:24.039607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:24.039607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:24.039607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:24.039607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:24.039607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:24.039607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:24.039607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:24.039607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:24.039607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:24.039607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:24.039607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.039607 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:24.040066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.040066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:24.040066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:24.040066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:24.040066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:24.040066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:24.040066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:24.040066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:24.040066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:24.040066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:24.040066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:24.040066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:24.040066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:24.040066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:24.040066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:24.040066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:24.040723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:43:24.040723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:43:24.040723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:43:24.040723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:24.040723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:24.040723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:24.040723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:24.040723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:24.040723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:24.040723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:24.040723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:24.040723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:24.040723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:24.040723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:24.040723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:24.040723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:24.040723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:24.040723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:24.041340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:24.041340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:24.041340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:24.041340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:24.041340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:24.041340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.041340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:24.050270 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:24.050736 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:24.050888 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:24.051049 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:24.051637 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:24.052312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.053242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:24.053242 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:24.053242 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:24.053242 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:24.053242 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:24.053242 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:24.053242 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.053242 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:24.054908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.054908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:43:24.054908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:24.054908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:24.054908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:24.054908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:24.054908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:24.054908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:24.054908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:24.054908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:24.054908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:24.054908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:43:24.054908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:24.054908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:43:24.054908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:43:24.055634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:24.055634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:43:24.055634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:24.055634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:43:24.055634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:43:24.055634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:24.055634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:43:24.055634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.055634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:43:24.056073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.056073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:24.056073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:24.056073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:24.056073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:24.056073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:43:24.056073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:43:24.056073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:24.056073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:24.056073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:24.056073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:24.056634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:24.056634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:24.056634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:24.056634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:24.056634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:24.056634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:24.056634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:24.056634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:24.056634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:24.056634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:24.056634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:24.056634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:24.056634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:24.056634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:24.057301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:24.057301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:24.057301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.057301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:24.058764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.058764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:24.058764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:24.058764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:24.058764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:24.058764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:24.058764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:24.058764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:24.058764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:24.058764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:24.058764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:24.058764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:24.058764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:24.058764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:24.058764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:24.058764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:24.058764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:43:24.059397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:43:24.059397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:24.059397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:24.059397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:24.059397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:24.059397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:24.059397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:24.059397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:24.059397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:24.059397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:24.059397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:24.059397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:24.059397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:24.059397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:24.059397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:24.059397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:24.059397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:24.059397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:24.059397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:24.060109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:24.060109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:24.060109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.060109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:24.125519 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:24.125931 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:24.126058 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:24.126249 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:24.127309 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:24.127743 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:24.127964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.128977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:24.128977 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:24.128977 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:24.128977 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:24.128977 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:24.128977 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:24.128977 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.128977 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:24.130463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.130463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:24.130463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:24.130463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:24.130463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:24.130463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:43:24.130463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:24.130463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:24.130463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:24.130463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:24.130463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:24.130463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:24.130921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:24.130921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:24.130921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:24.130921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:24.130921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:24.130921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:24.130921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:24.130921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:24.130921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:24.130921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:24.130921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:24.130921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:24.130921 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:24.131364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.131364 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:24.132096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.132096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:24.132096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:24.132096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:24.132096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:24.132096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:24.132096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:24.132096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:24.132096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:24.132096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:24.132096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:24.132096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:24.132096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:24.132096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:24.132096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:24.132096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:24.132608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:43:24.132608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:24.132608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:24.132608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:24.132608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:24.132608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:24.132608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:24.132608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:24.132608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:24.132608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:24.132608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:24.132608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:24.132608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:24.132608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:24.132608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:24.132608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:24.132608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:24.132608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:24.132608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:24.133229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:24.133229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:24.133229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:28.793859 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:28.794813 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:28.794813 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:28.795036 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:28.829467 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:28.829799 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:28.830711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:28.830711 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:28.830711 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:28.830711 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:28.830711 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:28.830711 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:28.830711 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:28.830711 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:28.831734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:28.831734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:28.831734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:28.831734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:28.831734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:28.831734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:43:28.831734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:28.831734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:28.831734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:28.831734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:28.831734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:28.831734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:28.832395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:28.832395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:28.832395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:28.832395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:28.832395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:28.832395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:28.832395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:28.832395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:28.832395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:28.832395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:28.832395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:28.832395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:28.832395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:28.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:28.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:28.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:28.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:28.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:28.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:28.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:28.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:28.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:28.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:28.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:28.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:28.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:28.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:28.833054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:28.833812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:28.833812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:28.833812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:28.833812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:43:28.833812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:28.833812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:28.833812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:28.833812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:28.833812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:28.833812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:28.833812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:28.833812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:28.833812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:28.833812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:28.833812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:28.833812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:28.833812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:28.833812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:28.833812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:28.834638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:28.834638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:28.834638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:28.834638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:28.834638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:28.834638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:29.007206 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:29.007785 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:29.007974 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:29.009797 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:29.009797 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:29.009797 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.010245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:29.010245 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:29.010245 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:29.010245 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:29.010245 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:29.010245 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:29.010245 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.010245 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:29.011786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.011786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:29.011786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:29.011786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:29.011786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:29.011786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:43:29.011786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:29.011786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:29.011786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:29.011786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:29.011786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:29.011786 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:29.012183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:29.012183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:29.012183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:29.012183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:29.012183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:29.012183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:29.012183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:29.012183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:29.012183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:29.012183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:29.012183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:29.012183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:29.012183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:29.012560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.012560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:29.013399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.013399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:29.013399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:29.013399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:29.013399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:29.013399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:29.013399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:29.013399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:29.013399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:29.013399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:29.013399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:29.013399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:29.013399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:29.013399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:29.013399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:29.013399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:29.014004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:43:29.014004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:29.014004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:29.014004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:29.014004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:29.014004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:29.014004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:29.014004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:29.014004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:29.014004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:29.014004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:29.014004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:29.014004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:29.014004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:29.014004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:29.014004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:29.014004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:29.014004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:29.014004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:29.014502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:29.014502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.014502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:29.030222 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:29.030626 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:29.031590 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:29.031590 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:43:29.031590 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:29.064873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.064873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:29.064873 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:29.064873 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:29.064873 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:29.064873 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:29.064873 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:29.064873 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.064873 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:29.064873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.064873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:43:29.064873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:29.064873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:29.064873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:29.064873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:29.064873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:29.064873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:29.065489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:29.065489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:29.065489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:29.065489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:43:29.065489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:29.065489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:43:29.065489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:43:29.065489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:29.065489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:43:29.065489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:29.065489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:43:29.065489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:43:29.065489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:29.065489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:43:29.065489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.065489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:43:29.066385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.066385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:29.066385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:29.066385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:29.066385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:29.066385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:43:29.066385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:43:29.066385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:43:29.066385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:29.066385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:29.066385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:29.066385 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:29.067566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:29.067566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:29.067566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:29.067566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:29.067566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:29.067566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:29.067566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:29.067566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:29.067566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:29.067566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:29.067566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:29.067566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:29.067566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:29.067566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:29.067991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:29.067991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:29.067991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.067991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:29.067991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.067991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:29.067991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:29.067991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:29.067991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:29.067991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:29.067991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:29.067991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:29.067991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:29.067991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:29.069748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:29.069748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:29.069748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:29.069748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:29.069748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:29.069748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:29.069748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:43:29.069748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:43:29.069748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:43:29.069748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:29.069748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:29.069748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:29.069748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:29.069748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:29.069748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:29.069748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:29.069748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:29.069748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:29.069748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:29.070565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:29.070565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:29.070565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:29.070565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:29.070565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:29.070565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:29.070565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:29.070565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:29.070565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:29.070565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:29.070565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.070565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:29.070565 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:29.070565 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:29.071032 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:29.071032 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:29.071032 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:29.071032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.071032 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:29.071032 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:29.071032 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:29.071032 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:29.071032 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:29.071032 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:29.071032 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.071032 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:29.071415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.071415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:43:29.071415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:29.071415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:29.071415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:29.071415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:29.071415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:29.071415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:29.071415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:29.071415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:29.071415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:29.071415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:43:29.071415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:29.071415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:43:29.071415 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:43:29.071874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:29.071874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:43:29.071874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:29.071874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:43:29.071874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:43:29.071874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:29.071874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:43:29.071874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.071874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:43:29.071874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.071874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:29.071874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:29.071874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:29.071874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:29.072292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:43:29.072292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:43:29.072292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:29.072292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:29.072292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:29.072292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:29.072292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:29.072292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:29.072292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:29.072292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:29.072292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:29.072292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:29.072292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:29.072743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:29.072743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:29.072743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:29.072743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:29.072743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:29.072743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:29.072743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:29.072743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:29.072743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:29.072743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.072743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:29.072743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.073116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:29.073116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:29.073116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:29.073116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:29.073116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:29.073116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:29.073116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:29.073116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:29.073116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:29.073116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:29.073116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:29.073116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:29.073116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:29.073116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:29.073116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:29.073116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:43:29.073116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:43:29.073116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:29.073116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:29.073606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:29.073606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:29.073606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:29.073606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:29.073606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:29.073606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:29.073606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:29.073606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:29.073606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:29.073606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:29.073606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:29.073606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:29.073606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:29.073606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:29.073606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:29.073606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:29.073606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:29.073606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:29.073606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.073606 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:29.126015 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:29.126409 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:29.126585 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:29.126880 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:29.127580 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:29.128006 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:29.128278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.129359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:29.129359 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:29.129359 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:29.129359 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:29.129359 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:29.129359 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:29.129359 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.129359 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:29.130722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.130722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:29.130722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:29.130722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:29.130722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:29.130722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:43:29.130722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:29.130722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:29.130722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:29.130722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:29.130722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:29.130722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:29.131293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:29.131293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:29.131293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:29.131293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:29.131293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:29.131293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:29.131293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:29.131293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:29.131293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:29.131293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:29.131293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:29.131293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:29.131293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:29.131654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.131654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:29.132728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.132728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:29.132728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:29.132728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:29.132728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:29.132728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:29.132728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:29.132728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:29.132728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:29.132728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:29.132728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:29.132728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:29.132728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:29.132728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:29.132728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:29.132728 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:29.133215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:43:29.133215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:29.133215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:29.133215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:29.133215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:29.133215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:29.133215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:29.133215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:29.133215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:29.133215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:29.133215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:29.133215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:29.133215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:29.133215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:29.133215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:29.133215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:29.133215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:29.133215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:29.133215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:29.133750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:29.133750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:29.133750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:33.794344 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:33.795044 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:33.795186 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:33.796268 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:33.833810 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:33.834137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:33.835427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:33.835427 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:33.835427 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:33.835427 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:33.835427 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:33.835427 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:33.835427 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:33.835427 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:33.836205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:33.836205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:33.836205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:33.836205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:33.836205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:33.836205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:43:33.836205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:33.836205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:33.836205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:33.836205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:33.836205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:33.836205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:33.836603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:33.836603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:33.836603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:33.836603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:33.836603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:33.836603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:33.836603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:33.836603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:33.836603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:33.836603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:33.836603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:33.836603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:33.836603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:33.837011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:33.837011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:33.837011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:33.837011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:33.837011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:33.837011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:33.837011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:33.837011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:33.837011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:33.837011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:33.837011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:33.837011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:33.837011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:33.837011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:33.837011 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:33.837496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:33.837496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:33.837496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:33.837496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:43:33.837496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:33.837496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:33.837496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:33.837496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:33.837496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:33.837496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:33.837496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:33.837496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:33.837496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:33.837496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:33.837496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:33.837496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:33.837496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:33.837496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:33.837496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:33.838088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:33.838088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:33.838088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:33.838088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:33.838088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:33.838088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:34.007822 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:34.009387 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:34.009582 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:34.010762 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:34.011161 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:34.011404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.012362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:34.012362 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:34.012362 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:34.012362 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:34.012362 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:34.012362 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:34.012362 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.012362 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:34.013907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.013907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:34.013907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:34.013907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:34.013907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:34.013907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:43:34.013907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:34.013907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:34.013907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:34.013907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:34.013907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:34.013907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:34.014292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:34.014292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:34.014292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:34.014292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:34.014292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:34.014292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:34.014292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:34.014292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:34.014292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:34.014292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:34.014292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:34.014292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:34.014292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:34.014629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.014629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:34.015536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.015536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:34.015536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:34.015536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:34.015536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:34.015536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:34.015536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:34.015536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:34.015536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:34.015536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:34.015536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:34.015536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:34.015536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:34.015536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:34.015536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:34.015536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:34.016015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:43:34.016015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:34.016015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:34.016015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:34.016015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:34.016015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:34.016015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:34.016015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:34.016015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:34.016015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:34.016015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:34.016015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:34.016015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:34.016015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:34.016015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:34.016015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:34.016015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:34.016015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:34.016015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:34.016557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:34.016557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.016557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:34.030573 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:34.030950 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:34.031156 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:34.031332 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:43:34.031863 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:34.032185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.032999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:34.032999 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:34.032999 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:34.032999 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:34.032999 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:34.032999 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:34.032999 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.032999 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:34.034155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.034155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:43:34.034155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:34.034155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:34.034155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:34.034155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:34.034155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:34.034155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:34.034155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:34.034155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:34.034155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:34.034155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:43:34.034155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:34.034155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:43:34.034155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:43:34.034668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:34.034668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:43:34.034668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:34.034668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:43:34.034668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:43:34.034668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:34.034668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:43:34.034668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.034668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:43:34.035453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.035453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:34.035453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:34.035453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:34.035453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:34.035453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:43:34.035453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:43:34.035453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:43:34.035453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:34.035453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:34.035453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:34.035453 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:34.035878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:34.035878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:34.035878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:34.035878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:34.035878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:34.035878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:34.035878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:34.035878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:34.035878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:34.035878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:34.035878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:34.035878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:34.035878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:34.035878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:34.036273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:34.036273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:34.036273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.036273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:34.037062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.037062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:34.037062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:34.037062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:34.037062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:34.037062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:34.037062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:34.037062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:34.037062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:34.037062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:34.037062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:34.037062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:34.037062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:34.037062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:34.037062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:34.037062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:34.037543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:43:34.037543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:43:34.037543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:43:34.037543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:34.037543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:34.037543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:34.037543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:34.037543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:34.037543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:34.037543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:34.037543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:34.037543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:34.037543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:34.037543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:34.037543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:34.037543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:34.037543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:34.037543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:34.038051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:34.038051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:34.038051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:34.038051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:34.038051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:34.038051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.038051 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:34.051205 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:34.051476 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:34.051644 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:34.051870 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:34.052358 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:34.052696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.053478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:34.053478 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:34.053478 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:34.053478 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:34.053478 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:34.053478 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:34.053478 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.053478 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:34.054591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.054591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:43:34.054591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:34.054591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:34.054591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:34.054591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:34.054591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:34.054591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:34.054591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:34.054591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:34.054591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:34.054591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:43:34.054591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:34.054591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:43:34.054591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:43:34.055026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:34.055026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:43:34.055026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:34.055026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:43:34.055026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:43:34.055026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:34.055026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:43:34.055026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.055026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:43:34.056007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.056007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:34.056007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:34.056007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:34.056007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:34.056007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:43:34.056007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:43:34.056007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:34.056007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:34.056007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:34.056007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:34.056362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:34.056362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:34.056362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:34.056362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:34.056362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:34.056362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:34.056362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:34.056362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:34.056362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:34.056362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:34.056362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:34.056362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:34.056362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:34.056362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:34.056765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:34.056765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:34.056765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.056765 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:34.057633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.057633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:34.057633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:34.057633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:34.057633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:34.057633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:34.057633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:34.057633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:34.057633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:34.057633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:34.057633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:34.057633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:34.057633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:34.057633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:34.057633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:34.057633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:34.057633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:43:34.058145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:43:34.058145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:34.058145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:34.058145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:34.058145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:34.058145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:34.058145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:34.058145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:34.058145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:34.058145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:34.058145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:34.058145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:34.058145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:34.058145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:34.058145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:34.058145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:34.058145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:34.058145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:34.058145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:34.058663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:34.058663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:34.058663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.058663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:34.126612 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:34.126992 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:34.127189 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:34.127420 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:34.128072 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:34.128452 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:34.128709 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.129472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:34.129472 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:34.129472 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:34.129472 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:34.129472 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:34.129472 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:34.129472 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.129472 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:34.130785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.130785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:34.130785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:34.130785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:34.130785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:34.130785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:43:34.130785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:34.130785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:34.130785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:34.130785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:34.130785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:34.130785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:34.131375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:34.131375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:34.131375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:34.131375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:34.131375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:34.131375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:34.131375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:34.131375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:34.131375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:34.131375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:34.131375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:34.131375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:34.131375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:34.131977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.131977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:34.132490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.132490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:34.132490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:34.132490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:34.132490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:34.132490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:34.132490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:34.132490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:34.132490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:34.132490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:34.132490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:34.132490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:34.132490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:34.132490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:34.132490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:34.132490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:34.133392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:43:34.133392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:34.133392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:34.133392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:34.133392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:34.133392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:34.133392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:34.133392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:34.133392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:34.133392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:34.133392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:34.133392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:34.133392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:34.133392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:34.133392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:34.133392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:34.133392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:34.133392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:34.133392 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:34.134208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:34.134208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:34.134208 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:38.795116 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:38.795801 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:38.795952 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:38.797275 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:38.830467 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:38.830731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:38.831262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:38.831262 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:38.831262 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:38.831262 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:38.831262 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:38.831262 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:38.831262 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:38.831262 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:38.832720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:38.832720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:38.832720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:38.832720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:38.832720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:38.832720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:43:38.832720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:38.832720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:38.832720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:38.832720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:38.832720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:38.832720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:38.833406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:38.833406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:38.833406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:38.833406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:38.833406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:38.833406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:38.833406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:38.833406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:38.833406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:38.833406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:38.833406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:38.833406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:38.833406 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:38.834038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:38.834038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:38.834536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:38.834536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:38.834536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:38.834536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:38.834536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:38.834536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:38.834536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:38.834536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:38.834536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:38.834536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:38.834536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:38.834536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:38.834536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:38.834536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:38.834536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:38.834536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:38.835377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:43:38.835377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:38.835377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:38.835377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:38.835377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:38.835377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:38.835377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:38.835377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:38.835377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:38.835377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:38.835377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:38.835377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:38.835377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:38.835377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:38.835377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:38.835377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:38.835377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:38.835377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:38.835377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:38.836271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:38.836271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:38.836271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:39.009621 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:39.010230 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:39.010345 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:39.011323 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:39.011646 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:39.011991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.012757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:39.012757 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:39.012757 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:39.012757 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:39.012757 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:39.012757 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:39.012757 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.012757 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:39.013867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.013867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:39.013867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:39.013867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:39.013867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:39.013867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:43:39.013867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:39.013867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:39.013867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:39.013867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:39.013867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:39.013867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:39.014630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:39.014630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:39.014630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:39.014630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:39.014630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:39.014630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:39.014630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:39.014630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:39.014630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:39.014630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:39.014630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:39.014630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:39.014630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:39.015131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.015131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:39.015131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.015131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:39.015131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:39.015131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:39.015131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:39.015131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:39.015131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:39.015131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:39.015131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:39.015131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:39.015131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:39.015131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:39.015131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:39.015753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:39.015753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:39.015753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:39.015753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:43:39.015753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:39.015753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:39.015753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:39.015753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:39.015753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:39.015753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:39.015753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:39.015753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:39.015753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:39.015753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:39.015753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:39.015753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:39.015753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:39.015753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:39.015753 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:39.016499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:39.016499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:39.016499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:39.016499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:39.016499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.016499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:39.030920 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:39.031158 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:39.031260 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:39.031396 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:43:39.032792 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:39.032951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.033768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:39.033768 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:39.033768 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:39.033768 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:39.033768 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:39.033768 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:39.033768 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.033768 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:39.034582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.034582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:43:39.034582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:39.034582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:39.034582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:39.034582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:39.034582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:39.034582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:39.034582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:39.034582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:39.034582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:39.034582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:43:39.034582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:39.034582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:43:39.034582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:43:39.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:39.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:43:39.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:39.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:43:39.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:43:39.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:39.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:43:39.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.035525 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:43:39.036138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.036138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:39.036138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:39.036138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:39.036138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:39.036138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:43:39.036138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:43:39.036138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:43:39.036138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:39.036138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:39.036138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:39.036138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:39.036951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:39.036951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:39.036951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:39.036951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:39.036951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:39.036951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:39.036951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:39.036951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:39.036951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:39.036951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:39.036951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:39.036951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:39.036951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:39.036951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:39.037916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:39.037916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:39.037916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.037916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:39.037916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.037916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:39.037916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:39.037916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:39.037916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:39.037916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:39.037916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:39.037916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:39.037916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:39.037916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:39.038700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:39.038700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:39.038700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:39.038700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:39.038700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:39.038700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:39.038700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:43:39.038700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:43:39.038700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:43:39.038700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:39.038700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:39.038700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:39.038700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:39.038700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:39.038700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:39.038700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:39.038700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:39.038700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:39.038700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:39.039456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:39.039456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:39.039456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:39.039456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:39.039456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:39.039456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:39.039456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:39.039456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:39.039456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:39.039456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:39.039456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.039456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:39.051646 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:39.052002 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:39.052119 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:39.052249 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:39.052901 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:39.053312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.054191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:39.054191 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:39.054191 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:39.054191 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:39.054191 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:39.054191 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:39.054191 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.054191 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:39.055335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.055335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:43:39.055335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:39.055335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:39.055335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:39.055335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:39.055335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:39.055335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:39.055335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:39.055335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:39.055335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:39.055335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:43:39.055335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:39.055335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:43:39.055335 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:43:39.055903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:39.055903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:43:39.055903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:39.055903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:43:39.055903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:43:39.055903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:39.055903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:43:39.055903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.055903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:43:39.056686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.056686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:39.056686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:39.056686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:39.056686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:39.056686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:43:39.056686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:43:39.056686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:39.056686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:39.056686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:39.056686 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:39.057109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:39.057109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:39.057109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:39.057109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:39.057109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:39.057109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:39.057109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:39.057109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:39.057109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:39.057109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:39.057109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:39.057109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:39.057109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:39.057109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:39.057642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:39.057642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:39.057642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.057642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:39.058379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.058379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:39.058379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:39.058379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:39.058379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:39.058379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:39.058379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:39.058379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:39.058379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:39.058379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:39.058379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:39.058379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:39.058379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:39.058379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:39.058379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:39.058379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:39.058379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:43:39.059396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:43:39.059396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:39.059396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:39.059396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:39.059396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:39.059396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:39.059396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:39.059396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:39.059396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:39.059396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:39.059396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:39.059396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:39.059396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:39.059396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:39.059396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:39.059396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:39.059396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:39.059396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:39.059396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:39.060583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:39.060583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:39.060583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.060583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:39.127134 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:39.127485 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:39.127657 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:39.127921 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:39.128479 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:39.128925 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:39.129167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.130044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:39.130044 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:39.130044 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:39.130044 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:39.130044 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:39.130044 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:39.130044 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.130044 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:39.131314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.131314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:39.131314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:39.131314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:39.131314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:39.131314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:43:39.131314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:39.131314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:39.131314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:39.131314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:39.131314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:39.131314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:39.131708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:39.131708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:39.131708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:39.131708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:39.131708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:39.131708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:39.131708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:39.131708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:39.131708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:39.131708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:39.131708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:39.131708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:39.131708 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:39.132069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.132069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:39.132922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.132922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:39.132922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:39.132922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:39.132922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:39.132922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:39.132922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:39.132922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:39.132922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:39.132922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:39.132922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:39.132922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:39.132922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:39.132922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:39.132922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:39.132922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:39.133716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:43:39.133716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:39.133716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:39.133716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:39.133716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:39.133716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:39.133716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:39.133716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:39.133716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:39.133716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:39.133716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:39.133716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:39.133716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:39.133716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:39.133716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:39.133716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:39.133716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:39.133716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:39.133716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:39.134599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:39.134599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:39.134599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:43.795908 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:43.796551 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:43.796791 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:43.798073 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:43.831174 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:43.831575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:43.832498 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:43.832498 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:43.832498 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:43.832498 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:43.832498 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:43.832498 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:43.832498 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:43.832498 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:43.833261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:43.833261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:43.833261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:43.833261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:43.833261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:43.833261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:43:43.833261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:43.833261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:43.833261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:43.833261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:43.833261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:43.833261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:43.833875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:43.833875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:43.833875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:43.833875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:43.833875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:43.833875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:43.833875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:43.833875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:43.833875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:43.833875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:43.833875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:43.833875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:43.833875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:43.834469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:43.834469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:43.834469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:43.834469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:43.834469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:43.834469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:43.834469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:43.834469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:43.834469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:43.834469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:43.834469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:43.834469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:43.834469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:43.834469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:43.834469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:43.835134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:43.835134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:43.835134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:43.835134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:43:43.835134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:43.835134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:43.835134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:43.835134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:43.835134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:43.835134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:43.835134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:43.835134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:43.835134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:43.835134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:43.835134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:43.835134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:43.835134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:43.835134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:43.835134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:43.835970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:43.835970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:43.835970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:43.835970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:43.835970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:43.835970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:44.010336 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:44.010935 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:44.011138 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:44.012342 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:44.012752 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:44.012993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.014047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:44.014047 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:44.014047 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:44.014047 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:44.014047 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:44.014047 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:44.014047 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.014047 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:44.015505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.015505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:44.015505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:44.015505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:44.015505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:44.015505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:43:44.015505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:44.015505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:44.015505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:44.015505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:44.015505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:44.015505 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:44.015967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:44.015967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:44.015967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:44.015967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:44.015967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:44.015967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:44.015967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:44.015967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:44.015967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:44.015967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:44.015967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:44.015967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:44.015967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:44.016338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.016338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:44.017309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.017309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:44.017309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:44.017309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:44.017309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:44.017309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:44.017309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:44.017309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:44.017309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:44.017309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:44.017309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:44.017309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:44.017309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:44.017309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:44.017309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:44.017309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:44.017776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:43:44.017776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:44.017776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:44.017776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:44.017776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:44.017776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:44.017776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:44.017776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:44.017776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:44.017776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:44.017776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:44.017776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:44.017776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:44.017776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:44.017776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:44.017776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:44.017776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:44.017776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:44.017776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:44.018290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:44.018290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.018290 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:44.031336 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:44.031833 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:44.032087 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:44.032193 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:43:44.032771 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:44.033068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.033850 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:44.033850 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:44.033850 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:44.033850 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:44.033850 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:44.033850 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:44.033850 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.033850 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:44.035015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.035015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:43:44.035015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:44.035015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:44.035015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:44.035015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:44.035015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:44.035015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:44.035015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:44.035015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:44.035015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:44.035015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:43:44.035015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:44.035015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:43:44.035015 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:43:44.035666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:44.035666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:43:44.035666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:44.035666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:43:44.035666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:43:44.035666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:44.035666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:43:44.035666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.035666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:43:44.036544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.036544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:44.036544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:44.036544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:44.036544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:44.036544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:43:44.036544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:43:44.036544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:43:44.036544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:44.036544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:44.036544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:44.036544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:44.036985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:44.036985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:44.036985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:44.036985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:44.036985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:44.036985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:44.036985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:44.036985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:44.036985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:44.036985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:44.036985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:44.036985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:44.036985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:44.036985 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:44.037371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:44.037371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:44.037371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.037371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:44.038058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.038058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:44.038058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:44.038058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:44.038058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:44.038058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:44.038058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:44.038058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:44.038058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:44.038058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:44.038058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:44.038058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:44.038058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:44.038058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:44.038058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:44.038058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:44.039476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:43:44.039476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:43:44.039476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:43:44.039476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:44.039476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:44.039476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:44.039476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:44.039476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:44.039476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:44.039476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:44.039476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:44.039476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:44.039476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:44.039476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:44.039476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:44.039476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:44.039476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:44.039476 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:44.039990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:44.039990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:44.039990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:44.039990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:44.039990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:44.039990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.039990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:44.052134 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:44.052560 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:44.052785 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:44.052981 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:44.053393 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:44.053615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.054212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:44.054212 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:44.054212 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:44.054212 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:44.054212 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:44.054212 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:44.054212 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.054212 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:44.055362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.055362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:43:44.055362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:44.055362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:44.055362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:44.055362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:44.055362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:44.055362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:44.055362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:44.055362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:44.055362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:44.055362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:43:44.055362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:44.055362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:43:44.055362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:43:44.056070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:44.056070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:43:44.056070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:44.056070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:43:44.056070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:43:44.056070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:44.056070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:43:44.056070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.056070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:43:44.056837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.056837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:44.056837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:44.056837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:44.056837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:44.056837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:43:44.056837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:43:44.056837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:44.056837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:44.056837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:44.056837 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:44.057461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:44.057461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:44.057461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:44.057461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:44.057461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:44.057461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:44.057461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:44.057461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:44.057461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:44.057461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:44.057461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:44.057461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:44.057461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:44.057461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:44.058054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:44.058054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:44.058054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.058054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:44.058982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.058982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:44.058982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:44.058982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:44.058982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:44.058982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:44.058982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:44.058982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:44.058982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:44.058982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:44.058982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:44.058982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:44.058982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:44.058982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:44.058982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:44.058982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:44.058982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:43:44.059746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:43:44.059746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:44.059746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:44.059746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:44.059746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:44.059746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:44.059746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:44.059746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:44.059746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:44.059746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:44.059746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:44.059746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:44.059746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:44.059746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:44.059746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:44.059746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:44.059746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:44.059746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:44.059746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:44.060586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:44.060586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:44.060586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.060586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:44.127532 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:44.128006 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:44.128196 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:44.128494 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:44.129276 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:44.129750 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:44.129988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.130872 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:44.130872 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:44.130872 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:44.130872 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:44.130872 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:44.130872 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:44.130872 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.130872 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:44.132416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.132416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:44.132416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:44.132416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:44.132416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:44.132416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:43:44.132416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:44.132416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:44.132416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:44.132416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:44.132416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:44.132416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:44.133102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:44.133102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:44.133102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:44.133102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:44.133102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:44.133102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:44.133102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:44.133102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:44.133102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:44.133102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:44.133102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:44.133102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:44.133102 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:44.133766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.133766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:44.134304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.134304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:44.134304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:44.134304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:44.134304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:44.134304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:44.134304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:44.134304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:44.134304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:44.134304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:44.134304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:44.134304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:44.134304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:44.134304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:44.134304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:44.134304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:44.135149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:43:44.135149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:44.135149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:44.135149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:44.135149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:44.135149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:44.135149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:44.135149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:44.135149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:44.135149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:44.135149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:44.135149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:44.135149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:44.135149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:44.135149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:44.135149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:44.135149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:44.135149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:44.135149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:44.136115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:44.136115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:44.136115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:48.796627 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:48.797604 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:48.797604 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:48.798617 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:48.828836 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:48.829225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:48.830069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:48.830069 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:48.830069 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:48.830069 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:48.830069 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:48.830069 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:48.830069 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:48.830069 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:48.830863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.033 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:48.830863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:48.830863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:48.830863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:48.830863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:48.830863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:43:48.830863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:48.830863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:48.830863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:48.830863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:48.830863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:48.830863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:48.831307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:48.831307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:48.831307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:48.831307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:48.831307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:48.831307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:48.831307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:48.831307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:48.831307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:48.831307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:48.831307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:48.831307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:48.831307 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:48.831802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:48.831802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:48.831802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:48.831802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:48.831802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:48.831802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:48.831802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:48.831802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:48.831802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:48.831802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:48.831802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:48.831802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:48.831802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:48.831802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:48.831802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:48.832355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:48.832355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:48.832355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:48.832355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:43:48.832355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:48.832355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:48.832355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:48.832355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:48.832355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:48.832355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:48.832355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:48.832355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:48.832355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:48.832355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:48.832355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:48.832355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:48.832355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:48.832355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:48.832355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:48.833047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:48.833047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:48.833047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:48.833047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:48.833047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:48.833047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:49.011182 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:49.011558 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:49.011779 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:49.013016 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:49.013405 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:49.013785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.014669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:49.014669 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:49.014669 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:49.014669 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:49.014669 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:49.014669 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:49.014669 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.014669 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:49.016121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.016121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:49.016121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:49.016121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:49.016121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:49.016121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:43:49.016121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:49.016121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:49.016121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:49.016121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:49.016121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:49.016121 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:49.016706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:49.016706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:49.016706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:49.016706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:49.016706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:49.016706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:49.016706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:49.016706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:49.016706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:49.016706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:49.016706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:49.016706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:49.016706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:49.017261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.017261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:49.017759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.017759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:49.017759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:49.017759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:49.017759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:49.017759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:49.017759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:49.017759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:49.017759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:49.017759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:49.017759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:49.017759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:49.017759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:49.017759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:49.017759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:49.017759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:49.018626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:43:49.018626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:49.018626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:49.018626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:49.018626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:49.018626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:49.018626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:49.018626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:49.018626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:49.018626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:49.018626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:49.018626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:49.018626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:49.018626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:49.018626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:49.018626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:49.018626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:49.018626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:49.018626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:49.019509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:49.019509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.019509 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:49.031874 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:49.032238 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:49.032413 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:49.032591 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:43:49.033119 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:49.033431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.034168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:49.034168 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:49.034168 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:49.034168 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:49.034168 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:49.034168 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:49.034168 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.034168 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:49.035359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.035359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:43:49.035359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:49.035359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:49.035359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:49.035359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:49.035359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:49.035359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:49.035359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:49.035359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:49.035359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:49.035359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:43:49.035359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:49.035359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:43:49.035359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:43:49.036162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:49.036162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:43:49.036162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:49.036162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:43:49.036162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:43:49.036162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:49.036162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:43:49.036162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.036162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:43:49.036594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.036594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:49.036594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:49.036594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:49.036594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:49.036594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:43:49.036594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:43:49.036594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:43:49.036594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:49.036594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:49.036594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:49.036594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:49.037144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:49.037144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:49.037144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:49.037144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:49.037144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:49.037144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:49.037144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:49.037144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:49.037144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:49.037144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:49.037144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:49.037144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:49.037144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:49.037144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:49.037784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:49.037784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:49.037784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.037784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:49.038210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.038210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:49.038210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:49.038210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:49.038210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:49.038210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:49.038210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:49.038210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:49.038210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:49.038210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:49.038210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:49.038210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:49.038210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:49.038210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:49.038210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:49.038210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:49.039027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:43:49.039027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:43:49.039027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:43:49.039027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:49.039027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:49.039027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:49.039027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:49.039027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:49.039027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:49.039027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:49.039027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:49.039027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:49.039027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:49.039027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:49.039027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:49.039027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:49.039027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:49.039027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:49.039859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:49.039859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:49.039859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:49.039859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:49.039859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:49.039859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.039859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:49.052727 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:49.053039 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:49.053202 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:49.053420 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:49.053961 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:49.054296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.055089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:49.055089 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:49.055089 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:49.055089 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:49.055089 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:49.055089 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:49.055089 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.055089 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:49.056229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.056229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:43:49.056229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:49.056229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:49.056229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:49.056229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:49.056229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:49.056229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:49.056229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:49.056229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:49.056229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:49.056229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:43:49.056229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:49.056229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:43:49.056229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:43:49.056870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:49.056870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:43:49.056870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:49.056870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:43:49.056870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:43:49.056870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:49.056870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:43:49.056870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.056870 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:43:49.057566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.057566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:49.057566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:49.057566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:49.057566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:49.057566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:43:49.057566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:43:49.057566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:49.057566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:49.057566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:49.057566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:49.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:49.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:49.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:49.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:49.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:49.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:49.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:49.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:49.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:49.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:49.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:49.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:49.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:49.057922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:49.058313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:49.058313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:49.058313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.058313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:49.059410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.059410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:49.059410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:49.059410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:49.059410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:49.059410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:49.059410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:49.059410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:49.059410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:49.059410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:49.059410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:49.059410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:49.059410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:49.059410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:49.059410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:49.059410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:49.059410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:43:49.060030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:43:49.060030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:49.060030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:49.060030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:49.060030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:49.060030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:49.060030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:49.060030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:49.060030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:49.060030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:49.060030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:49.060030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:49.060030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:49.060030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:49.060030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:49.060030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:49.060030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:49.060030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:49.060030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:49.060547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:49.060547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:49.060547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.060547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:49.128032 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:49.128426 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:49.128644 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:49.128832 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:49.129353 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:49.129694 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:49.129972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.130768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:49.130768 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:49.130768 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:49.130768 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:49.130768 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:49.130768 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:49.130768 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.130768 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:49.132080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.132080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:49.132080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:49.132080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:49.132080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:49.132080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:43:49.132080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:49.132080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:49.132080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:49.132080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:49.132080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:49.132080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:49.132857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:49.132857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:49.132857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:49.132857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:49.132857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:49.132857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:49.132857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:49.132857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:49.132857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:49.132857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:49.132857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:49.132857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:49.132857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:49.133584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.133584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:49.133584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.133584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:49.133584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:49.133584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:49.133584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:49.133584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:49.133584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:49.133584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:49.133584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:49.133584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:49.133584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:49.133584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:49.133584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:49.134656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:49.134656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:49.134656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:49.134656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:43:49.134656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:49.134656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:49.134656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:49.134656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:49.134656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:49.134656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:49.134656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:49.134656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:49.134656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:49.134656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:49.134656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:49.134656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:49.134656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:49.134656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:49.134656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:49.135929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:49.135929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:49.135929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:49.135929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:49.135929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:49.135929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:53.797302 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:53.798253 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:53.798253 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:53.799500 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:53.832896 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:53.833255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:53.834178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:53.834178 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:53.834178 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:53.834178 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:53.834178 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:53.834178 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:53.834178 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:53.834178 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:53.834873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:53.834873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:53.834873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:53.834873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:53.834873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:53.834873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:43:53.834873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:53.834873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:53.834873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:53.834873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:53.834873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:53.834873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:53.835538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:53.835538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:53.835538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:53.835538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:53.835538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:53.835538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:53.835538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:53.835538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:53.835538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:53.835538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:53.835538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:53.835538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:53.835538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:53.836271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:53.836271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:53.836271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:53.836271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:53.836271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:53.836271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:53.836271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:53.836271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:53.836271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:53.836271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:53.836271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:53.836271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:53.836271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:53.836271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:53.836271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:53.837074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:53.837074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:53.837074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:53.837074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:43:53.837074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:53.837074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:53.837074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:53.837074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:53.837074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:53.837074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:53.837074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:53.837074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:53.837074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:53.837074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:53.837074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:53.837074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:53.837074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:53.837074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:53.837074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:53.838107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:53.838107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:53.838107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:53.838107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:53.838107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:53.838107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:54.011768 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:54.012393 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:54.012613 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:54.014322 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:54.014585 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:54.015042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.016107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:54.016107 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:54.016107 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:54.016107 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:54.016107 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:54.016107 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:54.016107 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.016107 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:54.017995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.017995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:54.017995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:54.017995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:54.017995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:54.017995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:43:54.017995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:54.017995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:54.017995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:54.017995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:54.017995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:54.017995 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:54.018581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:54.018581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:54.018581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:54.018581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:54.018581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:54.018581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:54.018581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:54.018581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:54.018581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:54.018581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:54.018581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:54.018581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:54.018581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:54.019214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.019214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:54.020137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.020137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:54.020137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:54.020137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:54.020137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:54.020137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:54.020137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:54.020137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:54.020137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:54.020137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:54.020137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:54.020137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:54.020137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:54.020137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:54.020137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:54.020137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:54.020915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:43:54.020915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:54.020915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:54.020915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:54.020915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:54.020915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:54.020915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:54.020915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:54.020915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:54.020915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:54.020915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:54.020915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:54.020915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:54.020915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:54.020915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:54.020915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:54.020915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:54.020915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:54.020915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:54.021764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:54.021764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.021764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:54.032252 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:54.032655 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:54.032984 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:54.033128 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:43:54.035295 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:54.035295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.035295 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:54.035295 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:54.035295 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:54.035295 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:54.035295 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:54.035295 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:54.035295 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.035295 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:54.036910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.036910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:43:54.036910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:54.036910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:54.036910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:54.036910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:54.036910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:54.036910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:54.036910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:54.036910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:54.036910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:54.036910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:43:54.036910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:54.036910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:43:54.036910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:43:54.037801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:54.037801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:43:54.037801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:54.037801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:43:54.037801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:43:54.037801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:54.037801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:43:54.037801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.037801 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:43:54.038260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.038260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:54.038260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:54.038260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:54.038260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:54.038260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:43:54.038260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:43:54.038260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:43:54.038260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:54.038260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:54.038260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:54.038260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:54.038923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:54.038923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:54.038923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:54.038923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:54.038923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:54.038923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:54.038923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:54.038923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:54.038923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:54.038923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:54.038923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:54.038923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:54.038923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:54.038923 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:54.039591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:54.039591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:54.039591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.039591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:54.039932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.039932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:54.039932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:54.039932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:54.039932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:54.039932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:54.039932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:54.039932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:54.039932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:54.039932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:54.039932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:54.039932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:54.039932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:54.039932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:54.039932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:54.039932 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:54.040670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:43:54.040670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:43:54.040670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:43:54.040670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:54.040670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:54.040670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:54.040670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:54.040670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:54.040670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:54.040670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:54.040670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:54.040670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:54.040670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:54.040670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:54.040670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:54.040670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:54.040670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:54.040670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:54.041556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:54.041556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:54.041556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:54.041556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:54.041556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:54.041556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.041556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:54.053133 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:54.053395 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:54.053511 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:54.053756 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:54.054218 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:54.054554 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.055350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:54.055350 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:54.055350 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:54.055350 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:54.055350 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:54.055350 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:54.055350 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.055350 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:54.056625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.056625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:43:54.056625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:54.056625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:54.056625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:54.056625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:54.056625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:54.056625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:54.056625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:54.056625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:54.056625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:54.056625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:43:54.056625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:54.056625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:43:54.056625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:43:54.057340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:54.057340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:43:54.057340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:54.057340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:43:54.057340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:43:54.057340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:54.057340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:43:54.057340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.057340 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:43:54.058471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.058471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:54.058471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:54.058471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:54.058471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:54.058471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:43:54.058471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:43:54.058471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:54.058471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:54.058471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:54.058471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:54.059044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:54.059044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:54.059044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:54.059044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:54.059044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:54.059044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:54.059044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:54.059044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:54.059044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:54.059044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:54.059044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:54.059044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:54.059044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:54.059044 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:54.059639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:54.059639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:54.059639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.059639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:54.060316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.060316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:54.060316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:54.060316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:54.060316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:54.060316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:54.060316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:54.060316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:54.060316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:54.060316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:54.060316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:54.060316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:54.060316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:54.060316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:54.060316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:54.060316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:54.060316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:43:54.061192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:43:54.061192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:54.061192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:54.061192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:54.061192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:54.061192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:54.061192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:54.061192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:54.061192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:54.061192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:54.061192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:54.061192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:54.061192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:54.061192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:54.061192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:54.061192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:54.061192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:54.061192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:54.061192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:54.062041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:54.062041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:54.062041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.062041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:54.128510 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:54.129070 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:54.129257 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:54.129524 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:54.130059 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:54.130398 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:54.130649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.131428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:54.131428 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:54.131428 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:54.131428 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:54.131428 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:54.131428 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:54.131428 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.131428 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:54.132941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.132941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:54.132941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:54.132941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:54.132941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:54.132941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:43:54.132941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:54.132941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:54.132941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:54.132941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:54.132941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:54.132941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:54.133555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:54.133555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:54.133555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:54.133555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:54.133555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:54.133555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:54.133555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:54.133555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:54.133555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:54.133555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:54.133555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:54.133555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:54.133555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:54.134244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.134244 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:54.134601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.134601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:54.134601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:54.134601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:54.134601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:54.134601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:54.134601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:54.134601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:54.134601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:54.134601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:54.134601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:54.134601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:54.134601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:54.134601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:54.134601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:54.134601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:54.135553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:43:54.135553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:54.135553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:54.135553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:54.135553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:54.135553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:54.135553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:54.135553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:54.135553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:54.135553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:54.135553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:54.135553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:54.135553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:54.135553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:54.135553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:54.135553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:54.135553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:54.135553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:54.135553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:54.136418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:54.136418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:54.136418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:58.797967 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:58.798668 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:58.799018 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:58.800239 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:58.834120 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:58.834535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:58.835371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:58.835371 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:58.835371 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:58.835371 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:58.835371 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:58.835371 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:58.835371 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:58.835371 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:58.836202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:58.836202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:58.836202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:58.836202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:58.836202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:58.836202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:43:58.836202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:58.836202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:58.836202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:58.836202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:58.836202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:58.836202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:58.836719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:58.836719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:58.836719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:58.836719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:58.836719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:58.836719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:58.836719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:58.836719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:58.836719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:58.836719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:58.836719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:58.836719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:58.836719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:58.837282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:58.837282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:58.837282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:58.837282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:58.837282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:58.837282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:58.837282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:58.837282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:58.837282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:58.837282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:58.837282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:58.837282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:58.837282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:58.837282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:58.837282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:58.837848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:58.837848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:58.837848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:58.837848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:43:58.837848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:58.837848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:58.837848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:58.837848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:58.837848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:58.837848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:58.837848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:58.837848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:58.837848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:58.837848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:58.837848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:58.837848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:58.837848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:58.837848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:58.837848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:58.840241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:58.840241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:58.840241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:58.840241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:58.840241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:58.840241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:59.012502 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:59.013092 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:59.013390 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:59.014552 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:59.015083 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:59.015083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.016054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:59.016054 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:59.016054 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:59.016054 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:59.016054 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:59.016054 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:59.016054 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.016054 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:59.017645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.017645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:59.017645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:59.017645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:59.017645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:59.017645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:43:59.017645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:59.017645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:59.017645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:59.017645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:59.017645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:59.017645 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:59.018246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:59.018246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:59.018246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:59.018246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:59.018246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:59.018246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:59.018246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:59.018246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:59.018246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:59.018246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:59.018246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:59.018246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:59.018246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:59.018938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.018938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:59.019441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.019441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:59.019441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:59.019441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:59.019441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:59.019441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:59.019441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:59.019441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:59.019441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:59.019441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:59.019441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:59.019441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:59.019441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:59.019441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:59.019441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:59.019441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:59.020225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:43:59.020225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:59.020225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:59.020225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:59.020225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:59.020225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:59.020225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:59.020225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:59.020225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:59.020225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:59.020225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:59.020225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:59.020225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:59.020225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:59.020225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:59.020225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:59.020225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:59.020225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:59.020225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:59.021164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:59.021164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.021164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:59.032711 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:59.033013 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:59.033104 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:59.033235 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:43:59.034577 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:59.034907 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.036168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:59.036168 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:59.036168 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:59.036168 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:59.036168 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:59.036168 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:59.036168 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.036168 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:59.037338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.037338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:43:59.037338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:59.037338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:59.037338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:59.037338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:59.037338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:59.037338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:59.037338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:59.037338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:59.037338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:59.037338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:43:59.037338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:59.037338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:43:59.037338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:43:59.038077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:59.038077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:43:59.038077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:59.038077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:43:59.038077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:43:59.038077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:59.038077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:43:59.038077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.038077 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:43:59.039007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.039007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:59.039007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:59.039007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:59.039007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:59.039007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:43:59.039007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:43:59.039007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:43:59.039007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:59.039007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:59.039007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:59.039007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:59.040049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:59.040049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:59.040049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:59.040049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:59.040049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:59.040049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:59.040049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:59.040049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:59.040049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:59.040049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:59.040049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:59.040049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:59.040049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:59.040049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:59.040711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:59.040711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:59.040711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.040711 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:59.042092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.042092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:59.042092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:59.042092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:59.042092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:59.042092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:59.042092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:59.042092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:59.042092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:59.042092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:59.042092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:59.042092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:59.042092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:59.042092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:59.042092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:59.042092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:59.042520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:43:59.042520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:43:59.042520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:43:59.042520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:59.042520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:59.042520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:59.042520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:59.042520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:59.042520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:59.042520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:59.042520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:59.042520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:59.042520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:59.042520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:59.042520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:59.042520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:59.042520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:59.042520 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:59.043020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:59.043020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:59.043020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:59.043020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:59.043020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:59.043020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.043020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:59.053627 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:59.054058 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:59.054220 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:59.054408 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:59.054974 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:59.055328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.056098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:59.056098 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:59.056098 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:59.056098 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:59.056098 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:59.056098 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:59.056098 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.056098 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:59.057541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.057541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:43:59.057541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:59.057541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:59.057541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:59.057541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:59.057541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:59.057541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:59.057541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:59.057541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:43:59.057541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:59.057541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:43:59.057541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:59.057541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:43:59.057541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:43:59.058255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:59.058255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:43:59.058255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:59.058255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:43:59.058255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:43:59.058255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:59.058255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:43:59.058255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.058255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:43:59.059164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.059164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:59.059164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:59.059164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:59.059164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:59.059164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:43:59.059164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:43:59.059164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:59.059164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:59.059164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:59.059164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:59.059702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:59.059702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:59.059702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:59.059702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:59.059702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:59.059702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:59.059702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:59.059702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:59.059702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:59.059702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:59.059702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:59.059702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:59.059702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:59.059702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:59.060366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:59.060366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:59.060366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.060366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:59.061539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.061539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:59.061539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:59.061539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:59.061539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:59.061539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:59.061539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:59.061539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:59.061539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:59.061539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:59.061539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:59.061539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:59.061539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:59.061539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:59.061539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:59.061539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:59.061539 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:43:59.062281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:43:59.062281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:43:59.062281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:43:59.062281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:59.062281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:59.062281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:59.062281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:59.062281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:59.062281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:59.062281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:59.062281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:59.062281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:59.062281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:59.062281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:59.062281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:59.062281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:59.062281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:59.062281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:59.062281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:59.062951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:59.062951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:59.062951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.062951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:43:59.129069 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:43:59.129704 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:43:59.129961 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:43:59.130186 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:43:59.130708 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:59.131135 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:43:59.131362 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.132181 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:43:59.132181 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:59.132181 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:43:59.132181 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:43:59.132181 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:59.132181 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:43:59.132181 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.132181 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:43:59.133507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.133507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:43:59.133507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:59.133507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:43:59.133507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:59.133507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:43:59.133507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:59.133507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:59.133507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:59.133507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:59.133507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:59.133507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:59.134115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:43:59.134115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:59.134115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:43:59.134115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:59.134115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:43:59.134115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:43:59.134115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:59.134115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:43:59.134115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:59.134115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:43:59.134115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:43:59.134115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:59.134115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:43:59.134767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.134767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:43:59.134882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.134882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:43:59.134882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:43:59.134882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:43:59.134882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:43:59.134882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:43:59.134882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:59.134882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:43:59.134882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:43:59.134882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:59.134882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:43:59.134882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:43:59.134882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:43:59.134882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:43:59.134882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:43:59.134882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:43:59.135620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:43:59.135620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:43:59.135620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:59.135620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:59.135620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:43:59.135620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:43:59.135620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:43:59.135620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:43:59.135620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:43:59.135620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:43:59.135620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:43:59.135620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:43:59.135620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:43:59.135620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:43:59.135620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:43:59.135620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:43:59.135620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:43:59.135620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:43:59.135620 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:43:59.136558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:43:59.136558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:43:59.136558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:03.798638 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:03.799652 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:03.799652 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:03.801042 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:03.828653 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:03.829152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:03.830115 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:03.830115 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:03.830115 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:03.830115 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:03.830115 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:03.830115 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:03.830115 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:03.830115 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:03.830903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.030 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:03.830903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:03.830903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:03.830903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:03.830903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:03.830903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:44:03.830903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:03.830903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:03.830903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:03.830903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:03.830903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:03.830903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:03.831569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:03.831569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:03.831569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:03.831569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:03.831569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:03.831569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:03.831569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:03.831569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:03.831569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:03.831569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:03.831569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:03.831569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:03.831569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:03.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:03.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:03.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:03.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:03.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:03.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:03.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:03.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:03.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:03.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:03.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:03.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:03.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:03.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:03.832260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:03.833124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:03.833124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:03.833124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:03.833124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:44:03.833124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:03.833124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:03.833124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:03.833124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:03.833124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:03.833124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:03.833124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:03.833124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:03.833124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:03.833124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:03.833124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:03.833124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:03.833124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:03.833124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:03.833124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:03.834141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:03.834141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:03.834141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:03.834141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:03.834141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:03.834141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:04.013209 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:04.013804 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:04.014017 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:04.015244 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:04.015551 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:04.015864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.016656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:04.016656 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:04.016656 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:04.016656 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:04.016656 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:04.016656 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:04.016656 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.016656 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:04.018514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.018514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:04.018514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:04.018514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:04.018514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:04.018514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:44:04.018514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:04.018514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:04.018514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:04.018514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:04.018514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:04.018514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:04.019157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:04.019157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:04.019157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:04.019157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:04.019157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:04.019157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:04.019157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:04.019157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:04.019157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:04.019157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:04.019157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:04.019157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:04.019157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:04.019543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.019543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:04.020271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.020271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:04.020271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:04.020271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:04.020271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:04.020271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:04.020271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:04.020271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:04.020271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:04.020271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:04.020271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:04.020271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:04.020271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:04.020271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:04.020271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:04.020271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:04.020808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:44:04.020808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:04.020808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:04.020808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:04.020808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:04.020808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:04.020808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:04.020808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:04.020808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:04.020808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:04.020808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:04.020808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:04.020808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:04.020808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:04.020808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:04.020808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:04.020808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:04.020808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:04.020808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:04.021320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:04.021320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.021320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:04.033165 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:04.033600 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:04.033840 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:04.034072 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:44:04.034867 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:04.035123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.035856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:04.035856 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:04.035856 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:04.035856 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:04.035856 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:04.035856 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:04.035856 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.035856 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:04.037136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.037136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:44:04.037136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:04.037136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:04.037136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:04.037136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:04.037136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:04.037136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:04.037136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:04.037136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:04.037136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:04.037136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:44:04.037136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:04.037136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:44:04.037136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:44:04.038371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:04.038371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:44:04.038371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:04.038371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:44:04.038371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:44:04.038371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:04.038371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:44:04.038371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.038371 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:44:04.039086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.039086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:04.039086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:04.039086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:04.039086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:04.039086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:44:04.039086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:44:04.039086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:44:04.039086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:04.039086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:04.039086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:04.039086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:04.039914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:04.039914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:04.039914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:04.039914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:04.039914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:04.039914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:04.039914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:04.039914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:04.039914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:04.039914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:04.039914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:04.039914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:04.039914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:04.039914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:04.041526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:04.041526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:04.041526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.041526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:04.041526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.041526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:04.041526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:04.041526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:04.041526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:04.041526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:04.041526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:04.041526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:04.041526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:04.041526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:04.043148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:04.043148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:04.043148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:04.043148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:04.043148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:04.043148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:04.043148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:44:04.043148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:44:04.043148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:44:04.043148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:04.043148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:04.043148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:04.043148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:04.043148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:04.043148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:04.043148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:04.043148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:04.043148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:04.043148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:04.045777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:04.045777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:04.045777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:04.045777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:04.045777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:04.045777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:04.045777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:04.045777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:04.045777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:04.045777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:04.045777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.045777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:04.054044 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:04.054294 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:04.054464 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:04.054795 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:04.055468 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:04.055811 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.056702 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:04.056702 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:04.056702 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:04.056702 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:04.056702 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:04.056702 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:04.056702 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.056702 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:04.058283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.058283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:44:04.058283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:04.058283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:04.058283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:04.058283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:04.058283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:04.058283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:04.058283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:04.058283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:04.058283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:04.058283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:44:04.058283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:04.058283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:44:04.058283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:44:04.059367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:04.059367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:44:04.059367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:04.059367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:44:04.059367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:44:04.059367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:04.059367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:44:04.059367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.059367 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:44:04.059957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.059957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:04.059957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:04.059957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:04.059957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:04.059957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:44:04.059957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:44:04.059957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:04.059957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:04.059957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:04.059957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:04.060756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:04.060756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:04.060756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:04.060756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:04.060756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:04.060756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:04.060756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:04.060756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:04.060756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:04.060756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:04.060756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:04.060756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:04.060756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:04.060756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:04.061634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:04.061634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:04.061634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.061634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:04.062344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.062344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:04.062344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:04.062344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:04.062344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:04.062344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:04.062344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:04.062344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:04.062344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:04.062344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:04.062344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:04.062344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:04.062344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:04.062344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:04.062344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:04.062344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:04.062344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:44:04.064003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:44:04.064003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:04.064003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:04.064003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:04.064003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:04.064003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:04.064003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:04.064003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:04.064003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:04.064003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:04.064003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:04.064003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:04.064003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:04.064003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:04.064003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:04.064003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:04.064003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:04.064003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:04.064003 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:04.065467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:04.065467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:04.065467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.065467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:04.129830 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:04.130099 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:04.130267 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:04.130481 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:04.131212 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:04.131622 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:04.131887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.132764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:04.132764 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:04.132764 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:04.132764 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:04.132764 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:04.132764 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:04.132764 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.132764 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:04.133908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.133908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:04.133908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:04.133908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:04.133908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:04.133908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:44:04.133908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:04.133908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:04.133908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:04.133908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:04.133908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:04.133908 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:04.134446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:04.134446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:04.134446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:04.134446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:04.134446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:04.134446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:04.134446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:04.134446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:04.134446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:04.134446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:04.134446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:04.134446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:04.134446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:04.135422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.135422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:04.135422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.135422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:04.135422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:04.135422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:04.135422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:04.135422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:04.135422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:04.135422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:04.135422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:04.135422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:04.135422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:04.135422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:04.135422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:04.135892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:04.135892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:04.135892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:04.135892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:44:04.135892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:04.135892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:04.135892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:04.135892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:04.135892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:04.135892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:04.135892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:04.135892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:04.135892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:04.135892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:04.135892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:04.135892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:04.135892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:04.135892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:04.135892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:04.136394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:04.136394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:04.136394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:04.136394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:04.136394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:04.136394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:08.799297 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:08.800170 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:08.800381 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:08.801859 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:08.803962 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=240 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:44:08.837783 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:08.838584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:08.840012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:08.840012 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:08.840012 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:08.840012 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:08.840012 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:08.840012 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:08.840012 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:08.840012 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:08.841141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:08.841141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:08.841141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:08.841141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:08.841141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:08.841141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:44:08.841141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:08.841141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:08.841141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:08.841141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:08.841141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:08.841141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:08.841754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:08.841754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:08.841754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:08.841754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:08.841754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:08.841754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:08.841754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:08.841754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:08.841754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:08.841754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:08.841754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:08.841754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:08.841754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:08.842294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:08.842294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:08.842619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:08.842619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:08.842619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:08.842619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:08.842619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:08.842619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:08.842619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:08.842619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:08.842619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:08.842619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:08.842619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:08.842619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:08.842619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:08.842619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:08.842619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:08.842619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:08.843346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:44:08.843346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:08.843346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:08.843346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:08.843346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:08.843346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:08.843346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:08.843346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:08.843346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:08.843346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:08.843346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:08.843346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:08.843346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:08.843346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:08.843346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:08.843346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:08.843346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:08.843346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:08.843346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:08.844184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:08.844184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:08.844184 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:09.013910 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:09.014584 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:09.014942 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:09.016265 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:09.016708 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:09.017046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.018012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:09.018012 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:09.018012 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:09.018012 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:09.018012 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:09.018012 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:09.018012 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.018012 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:09.019854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.019854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:09.019854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:09.019854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:09.019854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:09.019854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:44:09.019854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:09.019854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:09.019854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:09.019854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:09.019854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:09.019854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:09.020357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:09.020357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:09.020357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:09.020357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:09.020357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:09.020357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:09.020357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:09.020357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:09.020357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:09.020357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:09.020357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:09.020357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:09.020357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:09.020810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.020810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:09.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:09.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:09.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:09.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:09.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:09.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:09.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:09.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:09.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:09.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:09.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:09.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:09.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:09.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:09.021819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:09.022353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:44:09.022353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:09.022353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:09.022353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:09.022353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:09.022353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:09.022353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:09.022353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:09.022353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:09.022353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:09.022353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:09.022353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:09.022353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:09.022353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:09.022353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:09.022353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:09.022353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:09.022353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:09.022353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:09.023001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:09.023001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.023001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:09.033591 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:09.034023 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:09.034246 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:09.034485 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:44:09.035015 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:09.035242 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.035909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:09.035909 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:09.035909 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:09.035909 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:09.035909 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:09.035909 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:09.035909 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.035909 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:09.037288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.037288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:44:09.037288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:09.037288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:09.037288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:09.037288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:09.037288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:09.037288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:09.037288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:09.037288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:09.037288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:09.037288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:44:09.037288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:09.037288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:44:09.037288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:44:09.038319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:09.038319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:44:09.038319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:09.038319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:44:09.038319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:44:09.038319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:09.038319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:44:09.038319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.038319 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:44:09.039164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.039164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:09.039164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:09.039164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:09.039164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:09.039164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:44:09.039164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:44:09.039164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:44:09.039164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:09.039164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:09.039164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:09.039164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:09.040080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:09.040080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:09.040080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:09.040080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:09.040080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:09.040080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:09.040080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:09.040080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:09.040080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:09.040080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:09.040080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:09.040080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:09.040080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:09.040080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:09.042268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:09.042268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:09.042268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.042268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:09.042268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.042268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:09.042268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:09.042268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:09.042268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:09.042268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:09.042268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:09.042268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:09.042268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:09.042268 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:09.042856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:09.042856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:09.042856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:09.042856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:09.042856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:09.042856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:09.042856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:44:09.042856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:44:09.042856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:44:09.042856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:09.042856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:09.042856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:09.042856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:09.042856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:09.042856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:09.042856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:09.042856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:09.042856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:09.042856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:09.043578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:09.043578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:09.043578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:09.043578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:09.043578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:09.043578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:09.043578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:09.043578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:09.043578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:09.043578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:09.043578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.043578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:09.054436 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:09.054736 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:09.054875 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:09.055011 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:09.055538 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:09.056034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.056968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:09.056968 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:09.056968 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:09.056968 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:09.056968 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:09.056968 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:09.056968 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.056968 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:09.058135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.058135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:44:09.058135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:09.058135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:09.058135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:09.058135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:09.058135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:09.058135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:09.058135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:09.058135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:09.058135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:09.058135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:44:09.058135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:09.058135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:44:09.058135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:44:09.058899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:09.058899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:44:09.058899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:09.058899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:44:09.058899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:44:09.058899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:09.058899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:44:09.058899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.058899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:44:09.060096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.060096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:09.060096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:09.060096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:09.060096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:09.060096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:44:09.060096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:44:09.060096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:09.060096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:09.060096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:09.060096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:09.060751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:09.060751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:09.060751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:09.060751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:09.060751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:09.060751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:09.060751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:09.060751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:09.060751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:09.060751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:09.060751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:09.060751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:09.060751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:09.060751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:09.061573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:09.061573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:09.061573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.061573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:09.062360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.062360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:09.062360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:09.062360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:09.062360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:09.062360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:09.062360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:09.062360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:09.062360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:09.062360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:09.062360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:09.062360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:09.062360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:09.062360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:09.062360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:09.062360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:09.062360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:44:09.063120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:44:09.063120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:09.063120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:09.063120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:09.063120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:09.063120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:09.063120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:09.063120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:09.063120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:09.063120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:09.063120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:09.063120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:09.063120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:09.063120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:09.063120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:09.063120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:09.063120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:09.063120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:09.063120 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:09.063991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:09.063991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:09.063991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.063991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:09.130255 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:09.130769 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:09.130992 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:09.131327 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:09.132166 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:09.132577 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:09.132734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.133455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:09.133455 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:09.133455 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:09.133455 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:09.133455 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:09.133455 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:09.133455 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.133455 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:09.134981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.134981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:09.134981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:09.134981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:09.134981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:09.134981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:44:09.134981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:09.134981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:09.134981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:09.134981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:09.134981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:09.134981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:09.135603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:09.135603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:09.135603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:09.135603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:09.135603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:09.135603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:09.135603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:09.135603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:09.135603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:09.135603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:09.135603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:09.135603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:09.135603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:09.136173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.136173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:09.137100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.137100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:09.137100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:09.137100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:09.137100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:09.137100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:09.137100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:09.137100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:09.137100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:09.137100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:09.137100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:09.137100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:09.137100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:09.137100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:09.137100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:09.137100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:09.137812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:44:09.137812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:09.137812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:09.137812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:09.137812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:09.137812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:09.137812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:09.137812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:09.137812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:09.137812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:09.137812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:09.137812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:09.137812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:09.137812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:09.137812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:09.137812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:09.137812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:09.137812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:09.137812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:09.138609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:09.138609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:09.138609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:13.800077 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:13.800771 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:13.800993 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:13.802488 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:13.841578 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:13.841945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:13.842720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:13.842720 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:13.842720 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:13.842720 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:13.842720 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:13.842720 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:13.842720 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:13.842720 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:13.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:13.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:13.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:13.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:13.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:13.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:44:13.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:13.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:13.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:13.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:13.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:13.843475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:13.844042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:13.844042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:13.844042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:13.844042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:13.844042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:13.844042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:13.844042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:13.844042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:13.844042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:13.844042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:13.844042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:13.844042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:13.844042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:13.844849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:13.844849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:13.844849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:13.844849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:13.844849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:13.844849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:13.844849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:13.844849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:13.844849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:13.844849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:13.844849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:13.844849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:13.844849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:13.844849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:13.844849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:13.845486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:13.845486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:13.845486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:13.845486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:44:13.845486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:13.845486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:13.845486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:13.845486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:13.845486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:13.845486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:13.845486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:13.845486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:13.845486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:13.845486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:13.845486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:13.845486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:13.845486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:13.845486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:13.845486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:13.846481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:13.846481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:13.846481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:13.846481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:13.846481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:13.846481 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:14.014663 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:14.015346 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:14.015597 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:14.017386 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:14.017751 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:14.018064 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.019025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:14.019025 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:14.019025 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:14.019025 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:14.019025 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:14.019025 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:14.019025 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.019025 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:14.020949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.020949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:14.020949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:14.020949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:14.020949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:14.020949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:44:14.020949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:14.020949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:14.020949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:14.020949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:14.020949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:14.020949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:14.022642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:14.022642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:14.022642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:14.022642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:14.022642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:14.022642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:14.022642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:14.022642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:14.022642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:14.022642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:14.022642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:14.022642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:14.022642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:14.024732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.024732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:14.024732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.024732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:14.024732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:14.024732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:14.024732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:14.024732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:14.024732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:14.024732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:14.024732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:14.024732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:14.024732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:14.024732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:14.024732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:14.025648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:14.025648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:14.025648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:14.025648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:44:14.025648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:14.025648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:14.025648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:14.025648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:14.025648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:14.025648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:14.025648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:14.025648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:14.025648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:14.025648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:14.025648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:14.025648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:14.025648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:14.025648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:14.025648 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:14.026265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:14.026265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:14.026265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:14.026265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:14.026265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.026265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:14.034087 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:14.034432 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:14.034567 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:14.034797 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:44:14.035567 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:14.036004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.036879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:14.036879 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:14.036879 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:14.036879 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:14.036879 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:14.036879 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:14.036879 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.036879 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:14.037922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.037922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:44:14.037922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:14.037922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:14.037922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:14.037922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:14.037922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:14.037922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:14.037922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:14.037922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:14.037922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:14.037922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:44:14.037922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:14.037922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:44:14.037922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:44:14.039084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:14.039084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:44:14.039084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:14.039084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:44:14.039084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:44:14.039084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:14.039084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:44:14.039084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.039084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:44:14.039084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.039084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:14.039084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:14.039084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:14.039084 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:14.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:44:14.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:44:14.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:44:14.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:14.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:14.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:14.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:14.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:14.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:14.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:14.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:14.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:14.040410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:14.040936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:14.040936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:14.040936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:14.040936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:14.040936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:14.040936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:14.040936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:14.040936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:14.040936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:14.040936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:14.040936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.040936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:14.041252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.041252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:14.041252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:14.041252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:14.041252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:14.041252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:14.041252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:14.041252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:14.041252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:14.041252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:14.041252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:14.041252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:14.041252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:14.041252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:14.041252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:14.041252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:14.041653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:44:14.041653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:44:14.041653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:44:14.041653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:14.041653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:14.041653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:14.041653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:14.041653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:14.041653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:14.041653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:14.041653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:14.041653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:14.041653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:14.041653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:14.041653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:14.041653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:14.041653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:14.041653 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:14.043027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:14.043027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:14.043027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:14.043027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:14.043027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:14.043027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.043027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:14.054905 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:14.055227 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:14.055362 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:14.055570 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:14.056089 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:14.056466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.057336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:14.057336 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:14.057336 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:14.057336 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:14.057336 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:14.057336 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:14.057336 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.057336 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:14.059092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.059092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:44:14.059092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:14.059092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:14.059092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:14.059092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:14.059092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:14.059092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:14.059092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:14.059092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:14.059092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:14.059092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:44:14.059092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:14.059092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:44:14.059092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:44:14.061278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:14.061278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:44:14.061278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:14.061278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:44:14.061278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:44:14.061278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:14.061278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:44:14.061278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.061278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:44:14.061278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.061278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:14.061278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:14.061278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:14.061278 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:14.062484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:44:14.062484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:44:14.062484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:14.062484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:14.062484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:14.062484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:14.062484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:14.062484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:14.062484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:14.062484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:14.062484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:14.062484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:14.062484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:14.063309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:14.063309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:14.063309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:14.063309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:14.063309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:14.063309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:14.063309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:14.063309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:14.063309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:14.063309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.063309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:14.064387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.064387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:14.064387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:14.064387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:14.064387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:14.064387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:14.064387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:14.064387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:14.064387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:14.064387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:14.064387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:14.064387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:14.064387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:14.064387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:14.064387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:14.064387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:14.064387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:44:14.064924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:44:14.064924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:14.064924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:14.064924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:14.064924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:14.064924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:14.064924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:14.064924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:14.064924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:14.064924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:14.064924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:14.064924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:14.064924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:14.064924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:14.064924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:14.064924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:14.064924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:14.064924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:14.064924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:14.065272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:14.065272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:14.065272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.065272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:14.130933 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:14.131480 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:14.131688 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:14.132106 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:14.132570 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:14.132950 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:14.133176 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.134000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:14.134000 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:14.134000 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:14.134000 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:14.134000 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:14.134000 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:14.134000 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.134000 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:14.135546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.135546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:14.135546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:14.135546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:14.135546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:14.135546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:44:14.135546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:14.135546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:14.135546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:14.135546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:14.135546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:14.135546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:14.136236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:14.136236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:14.136236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:14.136236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:14.136236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:14.136236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:14.136236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:14.136236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:14.136236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:14.136236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:14.136236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:14.136236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:14.136236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:14.136863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.136863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:14.137299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.137299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:14.137299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:14.137299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:14.137299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:14.137299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:14.137299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:14.137299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:14.137299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:14.137299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:14.137299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:14.137299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:14.137299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:14.137299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:14.137299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:14.137299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:14.138089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:44:14.138089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:14.138089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:14.138089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:14.138089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:14.138089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:14.138089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:14.138089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:14.138089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:14.138089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:14.138089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:14.138089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:14.138089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:14.138089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:14.138089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:14.138089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:14.138089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:14.138089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:14.138089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:14.139131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:14.139131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:14.139131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:18.800751 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:18.801603 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:18.801603 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:18.802131 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:18.828702 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:18.829220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:18.830247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:18.830247 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:18.830247 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:18.830247 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:18.830247 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:18.830247 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:18.830247 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:18.830247 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:18.831172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.029 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:18.831172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:18.831172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:18.831172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:18.831172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:18.831172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:44:18.831172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:18.831172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:18.831172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:18.831172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:18.831172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:18.831172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:18.831804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:18.831804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:18.831804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:18.831804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:18.831804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:18.831804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:18.831804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:18.831804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:18.831804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:18.831804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:18.831804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:18.831804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:18.831804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:18.832341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:18.832341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:18.832341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:18.832341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:18.832341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:18.832341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:18.832341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:18.832341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:18.832341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:18.832341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:18.832341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:18.832341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:18.832341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:18.832341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:18.832341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:18.833155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:18.833155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:18.833155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:18.833155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:44:18.833155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:18.833155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:18.833155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:18.833155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:18.833155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:18.833155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:18.833155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:18.833155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:18.833155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:18.833155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:18.833155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:18.833155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:18.833155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:18.833155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:18.833155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:18.833956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:18.833956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:18.833956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:18.833956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:18.833956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:18.833956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:19.015551 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:19.015849 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:19.015912 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:19.017009 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:19.017258 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:19.017428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.018217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:19.018217 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:19.018217 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:19.018217 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:19.018217 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:19.018217 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:19.018217 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.018217 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:19.019777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.019777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:19.019777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:19.019777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:19.019777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:19.019777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:44:19.019777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:19.019777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:19.019777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:19.019777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:19.019777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:19.019777 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:19.020063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:19.020063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:19.020063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:19.020063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:19.020063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:19.020063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:19.020063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:19.020063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:19.020063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:19.020063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:19.020063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:19.020063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:19.020063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:19.020334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.020334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:19.021422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.021422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:19.021422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:19.021422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:19.021422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:19.021422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:19.021422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:19.021422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:19.021422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:19.021422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:19.021422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:19.021422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:19.021422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:19.021422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:19.021422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:19.021422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:19.021854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:44:19.021854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:19.021854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:19.021854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:19.021854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:19.021854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:19.021854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:19.021854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:19.021854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:19.021854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:19.021854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:19.021854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:19.021854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:19.021854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:19.021854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:19.021854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:19.021854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:19.021854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:19.021854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:19.022390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:19.022390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.022390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:19.034794 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:19.035063 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:19.035356 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:19.035356 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:44:19.036127 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:19.036346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.037018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:19.037018 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:19.037018 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:19.037018 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:19.037018 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:19.037018 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:19.037018 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.037018 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:19.038308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.038308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:44:19.038308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:19.038308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:19.038308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:19.038308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:19.038308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:19.038308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:19.038308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:19.038308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:19.038308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:19.038308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:44:19.038308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:19.038308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:44:19.038308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:44:19.038905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:19.038905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:44:19.038905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:19.038905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:44:19.038905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:44:19.038905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:19.038905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:44:19.038905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.038905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:44:19.039878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.039878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:19.039878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:19.039878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:19.039878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:19.039878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:44:19.039878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:44:19.039878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:44:19.039878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:19.039878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:19.039878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:19.039878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:19.040413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:19.040413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:19.040413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:19.040413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:19.040413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:19.040413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:19.040413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:19.040413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:19.040413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:19.040413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:19.040413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:19.040413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:19.040413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:19.040413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:19.040906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:19.040906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:19.040906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.040906 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:19.041599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.041599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:19.041599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:19.041599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:19.041599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:19.041599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:19.041599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:19.041599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:19.041599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:19.041599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:19.041599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:19.041599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:19.041599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:19.041599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:19.041599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:19.041599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:19.042113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:44:19.042113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:44:19.042113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:44:19.042113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:19.042113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:19.042113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:19.042113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:19.042113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:19.042113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:19.042113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:19.042113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:19.042113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:19.042113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:19.042113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:19.042113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:19.042113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:19.042113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:19.042113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:19.042713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:19.042713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:19.042713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:19.042713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:19.042713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:19.042713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.042713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:19.055382 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:19.055593 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:19.055866 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:19.056105 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:19.056862 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:19.057312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.058277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:19.058277 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:19.058277 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:19.058277 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:19.058277 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:19.058277 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:19.058277 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.058277 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:19.059611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.059611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:44:19.059611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:19.059611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:19.059611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:19.059611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:19.059611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:19.059611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:19.059611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:19.059611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:19.059611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:19.059611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:44:19.059611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:19.059611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:44:19.059611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:44:19.060260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:19.060260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:44:19.060260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:19.060260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:44:19.060260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:44:19.060260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:19.060260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:44:19.060260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.060260 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:44:19.061261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.061261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:19.061261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:19.061261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:19.061261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:19.061261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:44:19.061261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:44:19.061261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:19.061261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:19.061261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:19.061261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:19.061767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:19.061767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:19.061767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:19.061767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:19.061767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:19.061767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:19.061767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:19.061767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:19.061767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:19.061767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:19.061767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:19.061767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:19.061767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:19.061767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:19.062191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:19.062191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:19.062191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.062191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:19.063294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.063294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:19.063294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:19.063294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:19.063294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:19.063294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:19.063294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:19.063294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:19.063294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:19.063294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:19.063294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:19.063294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:19.063294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:19.063294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:19.063294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:19.063294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:19.063294 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:44:19.063933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:44:19.063933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:19.063933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:19.063933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:19.063933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:19.063933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:19.063933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:19.063933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:19.063933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:19.063933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:19.063933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:19.063933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:19.063933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:19.063933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:19.063933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:19.063933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:19.063933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:19.063933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:19.063933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:19.064624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:19.064624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:19.064624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.064624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:19.131532 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:19.131992 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:19.132113 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:19.132374 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:19.133188 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:19.133651 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:19.133945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.135030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:19.135030 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:19.135030 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:19.135030 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:19.135030 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:19.135030 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:19.135030 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.135030 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:19.136493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.136493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:19.136493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:19.136493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:19.136493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:19.136493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:44:19.136493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:19.136493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:19.136493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:19.136493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:19.136493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:19.136493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:19.137130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:19.137130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:19.137130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:19.137130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:19.137130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:19.137130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:19.137130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:19.137130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:19.137130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:19.137130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:19.137130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:19.137130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:19.137130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:19.137787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.137787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:19.138186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.138186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:19.138186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:19.138186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:19.138186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:19.138186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:19.138186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:19.138186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:19.138186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:19.138186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:19.138186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:19.138186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:19.138186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:19.138186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:19.138186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:19.138186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:19.139023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:44:19.139023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:19.139023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:19.139023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:19.139023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:19.139023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:19.139023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:19.139023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:19.139023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:19.139023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:19.139023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:19.139023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:19.139023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:19.139023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:19.139023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:19.139023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:19.139023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:19.139023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:19.139023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:19.139926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:19.139926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:19.139926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:23.801444 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:23.802453 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:23.802453 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:23.803607 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:23.839301 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:23.839699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:23.840718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:23.840718 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:23.840718 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:23.840718 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:23.840718 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:23.840718 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:23.840718 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:23.840718 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:23.841895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:23.841895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:23.841895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:23.841895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:23.841895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:23.841895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:44:23.841895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:23.841895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:23.841895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:23.841895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:23.841895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:23.841895 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:23.842732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:23.842732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:23.842732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:23.842732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:23.842732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:23.842732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:23.842732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:23.842732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:23.842732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:23.842732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:23.842732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:23.842732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:23.842732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:23.843461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:23.843461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:23.843461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:23.843461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:23.843461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:23.843461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:23.843461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:23.843461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:23.843461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:23.843461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:23.843461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:23.843461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:23.843461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:23.843461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:23.843461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:23.844418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:23.844418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:23.844418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:23.844418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:44:23.844418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:23.844418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:23.844418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:23.844418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:23.844418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:23.844418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:23.844418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:23.844418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:23.844418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:23.844418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:23.844418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:23.844418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:23.844418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:23.844418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:23.844418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:23.845447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:23.845447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:23.845447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:23.845447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:23.845447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:23.845447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:24.016102 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:24.016782 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:24.016976 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:24.018294 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:24.018625 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:24.018899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.019828 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:24.019828 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:24.019828 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:24.019828 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:24.019828 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:24.019828 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:24.019828 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.019828 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:24.021420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.021420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:24.021420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:24.021420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:24.021420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:24.021420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:44:24.021420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:24.021420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:24.021420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:24.021420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:24.021420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:24.021420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:24.022054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:24.022054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:24.022054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:24.022054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:24.022054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:24.022054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:24.022054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:24.022054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:24.022054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:24.022054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:24.022054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:24.022054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:24.022054 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:24.022615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.022615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:24.023154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.023154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:24.023154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:24.023154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:24.023154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:24.023154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:24.023154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:24.023154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:24.023154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:24.023154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:24.023154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:24.023154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:24.023154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:24.023154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:24.023154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:24.023154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:24.024271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:44:24.024271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:24.024271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:24.024271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:24.024271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:24.024271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:24.024271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:24.024271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:24.024271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:24.024271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:24.024271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:24.024271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:24.024271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:24.024271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:24.024271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:24.024271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:24.024271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:24.024271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:24.024271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:24.025247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:24.025247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.025247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:24.035361 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:24.035621 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:24.035795 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:24.035907 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:44:24.036425 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:24.036649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.037302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:24.037302 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:24.037302 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:24.037302 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:24.037302 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:24.037302 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:24.037302 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.037302 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:24.038513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.038513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:44:24.038513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:24.038513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:24.038513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:24.038513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:24.038513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:24.038513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:24.038513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:24.038513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:24.038513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:24.038513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:44:24.038513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:24.038513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:44:24.038513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:44:24.039018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:24.039018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:44:24.039018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:24.039018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:44:24.039018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:44:24.039018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:24.039018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:44:24.039018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.039018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:44:24.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:24.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:24.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:24.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:24.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:44:24.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:44:24.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:44:24.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:24.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:24.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:24.039967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:24.040599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:24.040599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:24.040599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:24.040599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:24.040599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:24.040599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:24.040599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:24.040599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:24.040599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:24.040599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:24.040599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:24.040599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:24.040599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:24.040599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:24.041904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:24.041904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:24.041904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.041904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:24.041904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.041904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:24.041904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:24.041904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:24.041904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:24.041904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:24.041904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:24.041904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:24.041904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:24.041904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:24.043436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:24.043436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:24.043436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:24.043436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:24.043436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:24.043436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:24.043436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:44:24.043436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:44:24.043436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:44:24.043436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:24.043436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:24.043436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:24.043436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:24.043436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:24.043436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:24.043436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:24.043436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:24.043436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:24.043436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:24.045217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:24.045217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:24.045217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:24.045217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:24.045217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:24.045217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:24.045217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:24.045217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:24.045217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:24.045217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:24.045217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.045217 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:24.056125 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:24.056612 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:24.056846 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:24.057076 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:24.057618 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:24.058024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.058879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:24.058879 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:24.058879 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:24.058879 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:24.058879 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:24.058879 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:24.058879 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.058879 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:24.060348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.060348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:44:24.060348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:24.060348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:24.060348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:24.060348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:24.060348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:24.060348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:24.060348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:24.060348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:24.060348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:24.060348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:44:24.060348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:24.060348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:44:24.060348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:44:24.061387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:24.061387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:44:24.061387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:24.061387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:44:24.061387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:44:24.061387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:24.061387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:44:24.061387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.061387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:44:24.061963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.061963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:24.061963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:24.061963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:24.061963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:24.061963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:44:24.061963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:44:24.061963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:24.061963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:24.061963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:24.061963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:24.062609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:24.062609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:24.062609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:24.062609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:24.062609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:24.062609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:24.062609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:24.062609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:24.062609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:24.062609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:24.062609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:24.062609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:24.062609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:24.062609 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:24.063245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:24.063245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:24.063245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.063245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:24.063780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.063780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:24.063780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:24.063780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:24.063780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:24.063780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:24.063780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:24.063780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:24.063780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:24.063780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:24.063780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:24.063780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:24.063780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:24.063780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:24.063780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:24.063780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:24.063780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:44:24.064292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:44:24.064292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:24.064292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:24.064292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:24.064292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:24.064292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:24.064292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:24.064292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:24.064292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:24.064292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:24.064292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:24.064292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:24.064292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:24.064292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:24.064292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:24.064292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:24.064292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:24.064292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:24.064292 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:24.064812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:24.064812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:24.064812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.064812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:24.132165 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:24.132780 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:24.132903 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:24.133120 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:24.133901 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:24.134329 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:24.134553 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.135612 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:24.135612 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:24.135612 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:24.135612 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:24.135612 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:24.135612 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:24.135612 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.135612 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:24.136662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.136662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:24.136662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:24.136662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:24.136662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:24.136662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:44:24.136662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:24.136662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:24.136662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:24.136662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:24.136662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:24.136662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:24.137029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:24.137029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:24.137029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:24.137029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:24.137029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:24.137029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:24.137029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:24.137029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:24.137029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:24.137029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:24.137029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:24.137029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:24.137029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:24.137357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.137357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:24.137688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.137688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:24.137688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:24.137688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:24.137688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:24.137688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:24.137688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:24.137688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:24.137688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:24.137688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:24.137688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:24.137688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:24.137688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:24.137688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:24.137688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:24.137688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:24.138119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:44:24.138119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:24.138119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:24.138119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:24.138119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:24.138119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:24.138119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:24.138119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:24.138119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:24.138119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:24.138119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:24.138119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:24.138119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:24.138119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:24.138119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:24.138119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:24.138119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:24.138119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:24.138119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:24.138582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:24.138582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:24.138582 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:28.802241 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:28.803104 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:28.803104 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:28.804348 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:28.840255 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:28.840640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:28.841547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:28.841547 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:28.841547 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:28.841547 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:28.841547 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:28.841547 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:28.841547 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:28.841547 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:28.842316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:28.842316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:28.842316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:28.842316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:28.842316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:28.842316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:44:28.842316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:28.842316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:28.842316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:28.842316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:28.842316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:28.842316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:28.843080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:28.843080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:28.843080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:28.843080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:28.843080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:28.843080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:28.843080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:28.843080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:28.843080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:28.843080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:28.843080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:28.843080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:28.843080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:28.843869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:28.843869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:28.843869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:28.843869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:28.843869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:28.843869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:28.843869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:28.843869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:28.843869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:28.843869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:28.843869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:28.843869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:28.843869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:28.843869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:28.843869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:28.845018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:28.845018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:28.845018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:28.845018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:44:28.845018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:28.845018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:28.845018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:28.845018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:28.845018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:28.845018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:28.845018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:28.845018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:28.845018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:28.845018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:28.845018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:28.845018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:28.845018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:28.845018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:28.845018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:28.846049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:28.846049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:28.846049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:28.846049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:28.846049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:28.846049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:29.017257 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:29.017904 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:29.018110 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:29.019264 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:29.019602 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:29.019834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.020501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:29.020501 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:29.020501 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:29.020501 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:29.020501 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:29.020501 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:29.020501 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.020501 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:29.022148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.022148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:29.022148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:29.022148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:29.022148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:29.022148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:44:29.022148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:29.022148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:29.022148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:29.022148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:29.022148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:29.022148 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:29.022809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:29.022809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:29.022809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:29.022809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:29.022809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:29.022809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:29.022809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:29.022809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:29.022809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:29.022809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:29.022809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:29.022809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:29.022809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:29.023332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.023332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:29.024173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.024173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:29.024173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:29.024173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:29.024173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:29.024173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:29.024173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:29.024173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:29.024173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:29.024173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:29.024173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:29.024173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:29.024173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:29.024173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:29.024173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:29.024173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:29.024953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:44:29.024953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:29.024953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:29.024953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:29.024953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:29.024953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:29.024953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:29.024953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:29.024953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:29.024953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:29.024953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:29.024953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:29.024953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:29.024953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:29.024953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:29.024953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:29.024953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:29.024953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:29.024953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:29.025561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:29.025561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.025561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:29.036117 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:29.036583 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:29.036842 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:29.037053 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:44:29.037497 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:29.037760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.038414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:29.038414 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:29.038414 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:29.038414 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:29.038414 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:29.038414 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:29.038414 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.038414 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:29.039718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.039718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:44:29.039718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:29.039718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:29.039718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:29.039718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:29.039718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:29.039718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:29.039718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:29.039718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:29.039718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:29.039718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:44:29.039718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:29.039718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:44:29.039718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:44:29.040689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:29.040689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:44:29.040689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:29.040689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:44:29.040689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:44:29.040689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:29.040689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:44:29.040689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.040689 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:44:29.041300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.041300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:29.041300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:29.041300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:29.041300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:29.041300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:44:29.041300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:44:29.041300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:44:29.041300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:29.041300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:29.041300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:29.041300 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:29.042104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:29.042104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:29.042104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:29.042104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:29.042104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:29.042104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:29.042104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:29.042104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:29.042104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:29.042104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:29.042104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:29.042104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:29.042104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:29.042104 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:29.042571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:29.042571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:29.042571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.042571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:29.043784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.043784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:29.043784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:29.043784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:29.043784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:29.043784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:29.043784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:29.043784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:29.043784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:29.043784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:29.043784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:29.043784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:29.043784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:29.043784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:29.043784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:29.043784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:29.044389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:44:29.044389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:44:29.044389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:44:29.044389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:29.044389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:29.044389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:29.044389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:29.044389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:29.044389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:29.044389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:29.044389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:29.044389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:29.044389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:29.044389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:29.044389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:29.044389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:29.044389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:29.044389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:29.044894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:29.044894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:29.044894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:29.044894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:29.044894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:29.044894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.044894 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:29.057092 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:29.057437 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:29.057623 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:29.057893 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:29.058421 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:29.058957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.059815 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:29.059815 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:29.059815 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:29.059815 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:29.059815 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:29.059815 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:29.059815 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.059815 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:29.061027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.061027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:44:29.061027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:29.061027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:29.061027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:29.061027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:29.061027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:29.061027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:29.061027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:29.061027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:29.061027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:29.061027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:44:29.061027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:29.061027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:44:29.061027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:44:29.061381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:29.061381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:44:29.061381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:29.061381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:44:29.061381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:44:29.061381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:29.061381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:44:29.061381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.061381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:44:29.062446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.062446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:29.062446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:29.062446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:29.062446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:29.062446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:44:29.062446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:44:29.062446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:29.062446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:29.062446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:29.062446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:29.062795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:29.062795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:29.062795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:29.062795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:29.062795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:29.062795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:29.062795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:29.062795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:29.062795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:29.062795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:29.062795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:29.062795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:29.062795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:29.062795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:29.063128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:29.063128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:29.063128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.063128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:29.064265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.064265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:29.064265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:29.064265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:29.064265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:29.064265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:29.064265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:29.064265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:29.064265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:29.064265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:29.064265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:29.064265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:29.064265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:29.064265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:29.064265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:29.064265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:29.064265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:44:29.064716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:44:29.064716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:29.064716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:29.064716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:29.064716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:29.064716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:29.064716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:29.064716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:29.064716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:29.064716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:29.064716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:29.064716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:29.064716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:29.064716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:29.064716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:29.064716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:29.064716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:29.064716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:29.064716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:29.065170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:29.065170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:29.065170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.065170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:29.133097 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:29.133558 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:29.133860 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:29.134133 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:29.134896 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:29.135401 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:29.135718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.136632 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:29.136632 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:29.136632 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:29.136632 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:29.136632 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:29.136632 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:29.136632 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.136632 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:29.137543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.137543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:29.137543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:29.137543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:29.137543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:29.137543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:44:29.137543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:29.137543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:29.137543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:29.137543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:29.137543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:29.137543 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:29.137905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:29.137905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:29.137905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:29.137905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:29.137905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:29.137905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:29.137905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:29.137905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:29.137905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:29.137905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:29.137905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:29.137905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:29.137905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:29.138210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.138210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:29.138705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.138705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:29.138705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:29.138705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:29.138705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:29.138705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:29.138705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:29.138705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:29.138705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:29.138705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:29.138705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:29.138705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:29.138705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:29.138705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:29.138705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:29.138705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:29.139086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:44:29.139086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:29.139086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:29.139086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:29.139086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:29.139086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:29.139086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:29.139086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:29.139086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:29.139086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:29.139086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:29.139086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:29.139086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:29.139086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:29.139086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:29.139086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:29.139086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:29.139086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:29.139086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:29.139528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:29.139528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:29.139528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:33.803000 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:33.803737 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:33.803737 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:33.804437 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:33.833642 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:33.834045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:33.834973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:33.834973 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:33.834973 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:33.834973 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:33.834973 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:33.834973 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:33.834973 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:33.834973 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:33.835707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:33.835707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:33.835707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:33.835707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:33.835707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:33.835707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:44:33.835707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:33.835707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:33.835707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:33.835707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:33.835707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:33.835707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:33.836248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:33.836248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:33.836248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:33.836248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:33.836248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:33.836248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:33.836248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:33.836248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:33.836248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:33.836248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:33.836248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:33.836248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:33.836248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:33.836822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:33.836822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:33.836822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:33.836822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:33.836822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:33.836822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:33.836822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:33.836822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:33.836822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:33.836822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:33.836822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:33.836822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:33.836822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:33.836822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:33.836822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:33.837631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:33.837631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:33.837631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:33.837631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:44:33.837631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:33.837631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:33.837631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:33.837631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:33.837631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:33.837631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:33.837631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:33.837631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:33.837631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:33.837631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:33.837631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:33.837631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:33.837631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:33.837631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:33.837631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:33.839378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:33.839378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:33.839378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:33.839378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:33.839378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:33.839378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:34.017954 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:34.018492 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:34.018802 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:34.019922 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:34.020224 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:34.020464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.021262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:34.021262 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:34.021262 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:34.021262 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:34.021262 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:34.021262 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:34.021262 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.021262 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:34.022216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.022216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:34.022216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:34.022216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:34.022216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:34.022216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:44:34.022216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:34.022216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:34.022216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:34.022216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:34.022216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:34.022216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:34.022598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:34.022598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:34.022598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:34.022598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:34.022598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:34.022598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:34.022598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:34.022598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:34.022598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:34.022598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:34.022598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:34.022598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:34.022598 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:34.022965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.022965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:34.023296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.023296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:34.023296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:34.023296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:34.023296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:34.023296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:34.023296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:34.023296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:34.023296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:34.023296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:34.023296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:34.023296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:34.023296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:34.023296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:34.023296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:34.023296 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:34.023718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:44:34.023718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:34.023718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:34.023718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:34.023718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:34.023718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:34.023718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:34.023718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:34.023718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:34.023718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:34.023718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:34.023718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:34.023718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:34.023718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:34.023718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:34.023718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:34.023718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:34.023718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:34.023718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:34.024198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:34.024198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.024198 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:34.036626 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:34.037317 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:34.037438 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:34.037707 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:44:34.038244 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:34.038527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.039344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:34.039344 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:34.039344 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:34.039344 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:34.039344 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:34.039344 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:34.039344 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.039344 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:34.040667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.040667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:44:34.040667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:34.040667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:34.040667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:34.040667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:34.040667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:34.040667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:34.040667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:34.040667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:34.040667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:34.040667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:44:34.040667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:34.040667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:44:34.040667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:44:34.041478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:34.041478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:44:34.041478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:34.041478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:44:34.041478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:44:34.041478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:34.041478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:44:34.041478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.041478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:44:34.042045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.042045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:34.042045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:34.042045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:34.042045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:34.042045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:44:34.042045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:44:34.042045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:44:34.042045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:34.042045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:34.042045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:34.042045 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:34.042629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:34.042629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:34.042629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:34.042629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:34.042629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:34.042629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:34.042629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:34.042629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:34.042629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:34.042629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:34.042629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:34.042629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:34.042629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:34.042629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:34.043446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:34.043446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:34.043446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.043446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:34.043446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.043446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:34.043446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:34.043446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:34.043446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:34.043446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:34.043446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:34.043446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:34.043446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:34.043446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:34.044310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:34.044310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:34.044310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:34.044310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:34.044310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:34.044310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:34.044310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:44:34.044310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:44:34.044310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:44:34.044310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:34.044310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:34.044310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:34.044310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:34.044310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:34.044310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:34.044310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:34.044310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:34.044310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:34.044310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:34.044865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:34.044865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:34.044865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:34.044865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:34.044865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:34.044865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:34.044865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:34.044865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:34.044865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:34.044865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:34.044865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.044865 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:34.057719 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:34.057991 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:34.058196 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:34.058388 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:34.058947 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:34.059465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.060451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:34.060451 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:34.060451 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:34.060451 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:34.060451 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:34.060451 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:34.060451 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.060451 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:34.061701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.061701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:44:34.061701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:34.061701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:34.061701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:34.061701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:34.061701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:34.061701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:34.061701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:34.061701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:34.061701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:34.061701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:44:34.061701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:34.061701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:44:34.061701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:44:34.062183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:34.062183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:44:34.062183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:34.062183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:44:34.062183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:44:34.062183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:34.062183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:44:34.062183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.062183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:44:34.063316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.063316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:34.063316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:34.063316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:34.063316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:34.063316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:44:34.063316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:44:34.063316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:34.063316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:34.063316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:34.063316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:34.063667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:34.063667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:34.063667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:34.063667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:34.063667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:34.063667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:34.063667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:34.063667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:34.063667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:34.063667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:34.063667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:34.063667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:34.063667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:34.063667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:34.064068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:34.064068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:34.064068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.064068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:34.065103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.065103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:34.065103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:34.065103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:34.065103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:34.065103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:34.065103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:34.065103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:34.065103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:34.065103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:34.065103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:34.065103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:34.065103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:34.065103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:34.065103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:34.065103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:34.065103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:44:34.065576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:44:34.065576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:34.065576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:34.065576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:34.065576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:34.065576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:34.065576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:34.065576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:34.065576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:34.065576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:34.065576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:34.065576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:34.065576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:34.065576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:34.065576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:34.065576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:34.065576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:34.065576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:34.065576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:34.066138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:34.066138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:34.066138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.066138 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:34.133735 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:34.134184 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:34.134363 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:34.134618 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:34.134961 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:34.135262 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:34.135432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.136027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:34.136027 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:34.136027 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:34.136027 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:34.136027 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:34.136027 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:34.136027 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.136027 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:34.137427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.137427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:34.137427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:34.137427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:34.137427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:34.137427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:44:34.137427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:34.137427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:34.137427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:34.137427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:34.137427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:34.137427 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:34.137800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:34.137800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:34.137800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:34.137800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:34.137800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:34.137800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:34.137800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:34.137800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:34.137800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:34.137800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:34.137800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:34.137800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:34.137800 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:34.138149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.138149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:34.139161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.139161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:34.139161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:34.139161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:34.139161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:34.139161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:34.139161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:34.139161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:34.139161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:34.139161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:34.139161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:34.139161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:34.139161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:34.139161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:34.139161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:34.139161 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:34.139595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:44:34.139595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:34.139595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:34.139595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:34.139595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:34.139595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:34.139595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:34.139595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:34.139595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:34.139595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:34.139595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:34.139595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:34.139595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:34.139595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:34.139595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:34.139595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:34.139595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:34.139595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:34.139595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:34.140132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:34.140132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:34.140132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:38.803906 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:38.804818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:38.804818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:38.806125 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:38.844233 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:38.844516 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:38.845892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:38.845892 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:38.845892 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:38.845892 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:38.845892 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:38.845892 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:38.845892 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:38.845892 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:38.846696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:38.846696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:38.846696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:38.846696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:38.846696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:38.846696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:44:38.846696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:38.846696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:38.846696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:38.846696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:38.846696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:38.846696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:38.847131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:38.847131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:38.847131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:38.847131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:38.847131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:38.847131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:38.847131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:38.847131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:38.847131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:38.847131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:38.847131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:38.847131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:38.847131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:38.847535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:38.847535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:38.848050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:38.848050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:38.848050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:38.848050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:38.848050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:38.848050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:38.848050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:38.848050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:38.848050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:38.848050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:38.848050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:38.848050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:38.848050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:38.848050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:38.848050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:38.848050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:38.848578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:44:38.848578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:38.848578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:38.848578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:38.848578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:38.848578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:38.848578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:38.848578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:38.848578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:38.848578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:38.848578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:38.848578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:38.848578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:38.848578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:38.848578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:38.848578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:38.848578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:38.848578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:38.848578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:38.849193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:38.849193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:38.849193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:39.018864 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:39.019544 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:39.019789 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:39.021882 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:39.021882 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:39.021882 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.022626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:39.022626 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:39.022626 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:39.022626 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:39.022626 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:39.022626 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:39.022626 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.022626 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:39.024834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.024834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:39.024834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:39.024834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:39.024834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:39.024834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:44:39.024834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:39.024834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:39.024834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:39.024834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:39.024834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:39.024834 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:39.025533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:39.025533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:39.025533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:39.025533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:39.025533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:39.025533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:39.025533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:39.025533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:39.025533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:39.025533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:39.025533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:39.025533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:39.025533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:39.026135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.026135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:39.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:39.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:39.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:39.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:39.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:39.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:39.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:39.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:39.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:39.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:39.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:39.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:39.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:39.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:39.026579 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:39.027469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:44:39.027469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:39.027469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:39.027469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:39.027469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:39.027469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:39.027469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:39.027469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:39.027469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:39.027469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:39.027469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:39.027469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:39.027469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:39.027469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:39.027469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:39.027469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:39.027469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:39.027469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:39.027469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:39.028712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:39.028712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.028712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:39.037669 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:39.038040 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:39.038159 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:39.038320 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:44:39.038884 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:39.039238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.039953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:39.039953 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:39.039953 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:39.039953 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:39.039953 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:39.039953 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:39.039953 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.039953 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:39.040637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.040637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:44:39.040637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:39.040637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:39.040637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:39.040637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:39.040637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:39.040637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:39.040637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:39.040637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:39.040637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:39.040637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:44:39.040637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:39.040637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:44:39.040637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:44:39.041660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:39.041660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:44:39.041660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:39.041660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:44:39.041660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:44:39.041660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:39.041660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:44:39.041660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.041660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:44:39.041660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.041660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:39.041660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:39.041660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:39.041660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:39.042308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:44:39.042308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:44:39.042308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:44:39.042308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:39.042308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:39.042308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:39.042308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:39.042308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:39.042308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:39.042308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:39.042308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:39.042308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:39.042308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:39.044333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:39.044333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:39.044333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:39.044333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:39.044333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:39.044333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:39.044333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:39.044333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:39.044333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:39.044333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:39.044333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.044333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:39.045515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.045515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:39.045515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:39.045515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:39.045515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:39.045515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:39.045515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:39.045515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:39.045515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:39.045515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:39.045515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:39.045515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:39.045515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:39.045515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:39.045515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:39.045515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:39.045942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:44:39.045942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:44:39.045942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:44:39.045942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:39.045942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:39.045942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:39.045942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:39.045942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:39.045942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:39.045942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:39.045942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:39.045942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:39.045942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:39.045942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:39.045942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:39.045942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:39.045942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:39.045942 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:39.046338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:39.046338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:39.046338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:39.046338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:39.046338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:39.046338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.046338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:39.058192 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:39.058528 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:39.058702 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:39.058844 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:39.059359 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:39.059746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.060428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:39.060428 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:39.060428 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:39.060428 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:39.060428 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:39.060428 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:39.060428 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.060428 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:39.061210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.061210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:44:39.061210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:39.061210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:39.061210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:39.061210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:39.061210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:39.061210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:39.061210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:39.061210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:39.061210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:39.061210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:44:39.061210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:39.061210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:44:39.061210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:44:39.062030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:39.062030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:44:39.062030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:39.062030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:44:39.062030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:44:39.062030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:39.062030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:44:39.062030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.062030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:44:39.062436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.062436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:39.062436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:39.062436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:39.062436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:39.062436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:44:39.062436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:44:39.062436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:39.062436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:39.062436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:39.062436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:39.063014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:39.063014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:39.063014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:39.063014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:39.063014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:39.063014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:39.063014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:39.063014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:39.063014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:39.063014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:39.063014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:39.063014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:39.063014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:39.063014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:39.063661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:39.063661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:39.063661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.063661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:39.063661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.063661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:39.063661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:39.063661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:39.063661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:39.063661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:39.063661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:39.063661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:39.063661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:39.063661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:39.064397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:39.064397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:39.064397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:39.064397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:39.064397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:39.064397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:39.064397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:44:39.064397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:44:39.064397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:39.064397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:39.064397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:39.064397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:39.064397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:39.064397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:39.064397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:39.064397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:39.064397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:39.064397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:39.064397 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:39.065672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:39.065672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:39.065672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:39.065672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:39.065672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:39.065672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:39.065672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:39.065672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:39.065672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:39.065672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.065672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:39.134554 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:39.135211 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:39.135406 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:39.135695 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:39.136383 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:39.136817 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:39.137062 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.137997 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:39.137997 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:39.137997 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:39.137997 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:39.137997 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:39.137997 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:39.137997 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.137997 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:39.139594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.139594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:39.139594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:39.139594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:39.139594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:39.139594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:44:39.139594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:39.139594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:39.139594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:39.139594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:39.139594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:39.139594 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:39.140257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:39.140257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:39.140257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:39.140257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:39.140257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:39.140257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:39.140257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:39.140257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:39.140257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:39.140257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:39.140257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:39.140257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:39.140257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:39.140853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.140853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:39.141431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.141431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:39.141431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:39.141431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:39.141431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:39.141431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:39.141431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:39.141431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:39.141431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:39.141431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:39.141431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:39.141431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:39.141431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:39.141431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:39.141431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:39.141431 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:39.142227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:44:39.142227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:39.142227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:39.142227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:39.142227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:39.142227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:39.142227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:39.142227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:39.142227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:39.142227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:39.142227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:39.142227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:39.142227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:39.142227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:39.142227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:39.142227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:39.142227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:39.142227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:39.142227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:39.143224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:39.143224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:39.143224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:43.804764 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:43.805725 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:43.805725 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:43.807042 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:43.852483 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:43.853187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:43.853845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:43.853845 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:43.853845 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:43.853845 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:43.853845 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:43.853845 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:43.853845 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:43.853845 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:43.854772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:43.854772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:43.854772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:43.854772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:43.854772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:43.854772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:44:43.854772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:43.854772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:43.854772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:43.854772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:43.854772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:43.854772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:43.855368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:43.855368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:43.855368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:43.855368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:43.855368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:43.855368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:43.855368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:43.855368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:43.855368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:43.855368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:43.855368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:43.855368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:43.855368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:43.855641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:43.855641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:43.855928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:43.855928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:43.855928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:43.855928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:43.855928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:43.855928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:43.855928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:43.855928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:43.855928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:43.855928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:43.855928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:43.855928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:43.855928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:43.855928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:43.855928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:43.855928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:43.856764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:44:43.856764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:43.856764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:43.856764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:43.856764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:43.856764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:43.856764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:43.856764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:43.856764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:43.856764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:43.856764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:43.856764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:43.856764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:43.856764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:43.856764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:43.856764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:43.856764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:43.856764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:43.856764 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:43.857359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:43.857359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:43.857359 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:44.019651 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:44.020304 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:44.020507 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:44.021776 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:44.022192 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:44.022376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.023368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:44.023368 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:44.023368 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:44.023368 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:44.023368 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:44.023368 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:44.023368 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.023368 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:44.024540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.024540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:44.024540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:44.024540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:44.024540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:44.024540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:44:44.024540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:44.024540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:44.024540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:44.024540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:44.024540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:44.024540 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:44.025132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:44.025132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:44.025132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:44.025132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:44.025132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:44.025132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:44.025132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:44.025132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:44.025132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:44.025132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:44.025132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:44.025132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:44.025132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:44.025767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.025767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:44.025767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.025767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:44.025767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:44.025767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:44.025767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:44.025767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:44.025767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:44.025767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:44.025767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:44.025767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:44.025767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:44.025767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:44.025767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:44.026378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:44.026378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:44.026378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:44.026378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:44:44.026378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:44.026378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:44.026378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:44.026378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:44.026378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:44.026378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:44.026378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:44.026378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:44.026378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:44.026378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:44.026378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:44.026378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:44.026378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:44.026378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:44.026378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:44.027224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:44.027224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:44.027224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:44.027224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:44.027224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.027224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:44.038299 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:44.038842 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:44.039155 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:44.039432 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:44:44.040124 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:44.040448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.041885 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:44.041885 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:44.041885 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:44.041885 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:44.041885 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:44.041885 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:44.041885 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.041885 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:44.043146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.043146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:44:44.043146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:44.043146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:44.043146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:44.043146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:44.043146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:44.043146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:44.043146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:44.043146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:44.043146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:44.043146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:44:44.043146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:44.043146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:44:44.043146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:44:44.043778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:44.043778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:44:44.043778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:44.043778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:44:44.043778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:44:44.043778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:44.043778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:44:44.043778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.043778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:44:44.044374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.044374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:44.044374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:44.044374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:44.044374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:44.044374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:44:44.044374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:44:44.044374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:44:44.044374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:44.044374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:44.044374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:44.044374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:44.045310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:44.045310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:44.045310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:44.045310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:44.045310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:44.045310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:44.045310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:44.045310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:44.045310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:44.045310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:44.045310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:44.045310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:44.045310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:44.045310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:44.046446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:44.046446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:44.046446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.046446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:44.046446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.046446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:44.046446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:44.046446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:44.046446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:44.046446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:44.046446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:44.046446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:44.046446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:44.046446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:44.047053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:44.047053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:44.047053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:44.047053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:44.047053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:44.047053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:44.047053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:44:44.047053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:44:44.047053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:44:44.047053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:44.047053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:44.047053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:44.047053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:44.047053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:44.047053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:44.047053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:44.047053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:44.047053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:44.047053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:44.047583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:44.047583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:44.047583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:44.047583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:44.047583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:44.047583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:44.047583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:44.047583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:44.047583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:44.047583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:44.047583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.047583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:44.058846 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:44.059318 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:44.059509 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:44.059755 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:44.060306 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:44.060734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.061731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:44.061731 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:44.061731 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:44.061731 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:44.061731 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:44.061731 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:44.061731 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.061731 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:44.062980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.062980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:44:44.062980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:44.062980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:44.062980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:44.062980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:44.062980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:44.062980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:44.062980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:44.062980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:44.062980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:44.062980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:44:44.062980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:44.062980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:44:44.062980 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:44:44.064096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:44.064096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:44:44.064096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:44.064096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:44:44.064096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:44:44.064096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:44.064096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:44:44.064096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.064096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:44:44.064521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.064521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:44.064521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:44.064521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:44.064521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:44.064521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:44:44.064521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:44:44.064521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:44.064521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:44.064521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:44.064521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:44.065026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:44.065026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:44.065026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:44.065026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:44.065026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:44.065026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:44.065026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:44.065026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:44.065026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:44.065026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:44.065026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:44.065026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:44.065026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:44.065026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:44.065567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:44.065567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:44.065567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.065567 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:44.066639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.066639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:44.066639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:44.066639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:44.066639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:44.066639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:44.066639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:44.066639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:44.066639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:44.066639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:44.066639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:44.066639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:44.066639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:44.066639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:44.066639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:44.066639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:44.066639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:44:44.068236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:44:44.068236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:44.068236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:44.068236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:44.068236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:44.068236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:44.068236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:44.068236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:44.068236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:44.068236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:44.068236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:44.068236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:44.068236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:44.068236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:44.068236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:44.068236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:44.068236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:44.068236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:44.068236 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:44.070153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:44.070153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:44.070153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.070153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:44.135177 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:44.135716 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:44.135911 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:44.136152 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:44.136881 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:44.137300 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:44.137536 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.138361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:44.138361 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:44.138361 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:44.138361 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:44.138361 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:44.138361 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:44.138361 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.138361 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:44.139774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.139774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:44.139774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:44.139774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:44.139774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:44.139774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:44:44.139774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:44.139774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:44.139774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:44.139774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:44.139774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:44.139774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:44.140306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:44.140306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:44.140306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:44.140306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:44.140306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:44.140306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:44.140306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:44.140306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:44.140306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:44.140306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:44.140306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:44.140306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:44.140306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:44.140978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.140978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:44.141311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.141311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:44.141311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:44.141311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:44.141311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:44.141311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:44.141311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:44.141311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:44.141311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:44.141311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:44.141311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:44.141311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:44.141311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:44.141311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:44.141311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:44.141311 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:44.141975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:44:44.141975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:44.141975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:44.141975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:44.141975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:44.141975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:44.141975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:44.141975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:44.141975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:44.141975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:44.141975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:44.141975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:44.141975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:44.141975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:44.141975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:44.141975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:44.141975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:44.141975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:44.141975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:44.143353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:44.143353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:44.143353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:48.805432 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:48.806406 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:48.806406 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:48.807597 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:48.843190 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:48.843580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:48.844258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:48.844258 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:48.844258 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:48.844258 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:48.844258 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:48.844258 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:48.844258 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:48.844258 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:48.845029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:48.845029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:48.845029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:48.845029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:48.845029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:48.845029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:44:48.845029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:48.845029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:48.845029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:48.845029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:48.845029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:48.845029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:48.845432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:48.845432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:48.845432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:48.845432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:48.845432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:48.845432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:48.845432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:48.845432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:48.845432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:48.845432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:48.845432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:48.845432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:48.845432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:48.845856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:48.845856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:48.845856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:48.845856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:48.845856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:48.845856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:48.845856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:48.845856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:48.845856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:48.845856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:48.845856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:48.845856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:48.845856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:48.845856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:48.845856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:48.846583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:48.846583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:48.846583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:48.846583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:44:48.846583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:48.846583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:48.846583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:48.846583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:48.846583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:48.846583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:48.846583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:48.846583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:48.846583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:48.846583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:48.846583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:48.846583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:48.846583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:48.846583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:48.846583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:48.848117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:48.848117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:48.848117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:48.848117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:48.848117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:48.848117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:49.020808 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:49.021383 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:49.021597 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:49.022644 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:49.022974 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:49.023215 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.024156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:49.024156 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:49.024156 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:49.024156 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:49.024156 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:49.024156 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:49.024156 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.024156 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:49.025732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.025732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:49.025732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:49.025732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:49.025732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:49.025732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:44:49.025732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:49.025732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:49.025732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:49.025732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:49.025732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:49.025732 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:49.026298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:49.026298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:49.026298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:49.026298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:49.026298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:49.026298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:49.026298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:49.026298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:49.026298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:49.026298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:49.026298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:49.026298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:49.026298 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:49.026836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.026836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:49.027656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.027656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:49.027656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:49.027656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:49.027656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:49.027656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:49.027656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:49.027656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:49.027656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:49.027656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:49.027656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:49.027656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:49.027656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:49.027656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:49.027656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:49.027656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:49.028912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:44:49.028912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:49.028912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:49.028912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:49.028912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:49.028912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:49.028912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:49.028912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:49.028912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:49.028912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:49.028912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:49.028912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:49.028912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:49.028912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:49.028912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:49.028912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:49.028912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:49.028912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:49.028912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:49.029590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:49.029590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.029590 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:49.038825 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:49.039112 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:49.039226 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:49.039362 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:44:49.039903 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:49.040219 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.040976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:49.040976 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:49.040976 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:49.040976 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:49.040976 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:49.040976 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:49.040976 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.040976 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:49.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:44:49.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:49.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:49.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:49.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:49.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:49.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:49.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:49.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:49.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:49.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:44:49.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:49.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:44:49.042210 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:44:49.042619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:49.042619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:44:49.042619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:49.042619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:44:49.042619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:44:49.042619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:49.042619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:44:49.042619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.042619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:44:49.043622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.043622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:49.043622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:49.043622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:49.043622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:49.043622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:44:49.043622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:44:49.043622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:44:49.043622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:49.043622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:49.043622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:49.043622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:49.044059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:49.044059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:49.044059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:49.044059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:49.044059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:49.044059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:49.044059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:49.044059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:49.044059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:49.044059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:49.044059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:49.044059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:49.044059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:49.044059 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:49.044365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:49.044365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:49.044365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.044365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:49.045619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.045619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:49.045619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:49.045619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:49.045619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:49.045619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:49.045619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:49.045619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:49.045619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:49.045619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:49.045619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:49.045619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:49.045619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:49.045619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:49.045619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:49.045619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:49.046025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:44:49.046025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:44:49.046025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:44:49.046025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:49.046025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:49.046025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:49.046025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:49.046025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:49.046025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:49.046025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:49.046025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:49.046025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:49.046025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:49.046025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:49.046025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:49.046025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:49.046025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:49.046025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:49.046444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:49.046444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:49.046444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:49.046444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:49.046444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:49.046444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.046444 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:49.059449 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:49.059969 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:49.059969 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:49.060120 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:49.060559 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:49.060955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.061629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:49.061629 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:49.061629 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:49.061629 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:49.061629 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:49.061629 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:49.061629 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.061629 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:49.062840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.062840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:44:49.062840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:49.062840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:49.062840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:49.062840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:49.062840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:49.062840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:49.062840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:49.062840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:49.062840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:49.062840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:44:49.062840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:49.062840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:44:49.062840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:44:49.063425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:49.063425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:44:49.063425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:49.063425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:44:49.063425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:44:49.063425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:49.063425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:44:49.063425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.063425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:44:49.064225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.064225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:49.064225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:49.064225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:49.064225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:49.064225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:44:49.064225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:44:49.064225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:49.064225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:49.064225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:49.064225 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:49.064532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:49.064532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:49.064532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:49.064532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:49.064532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:49.064532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:49.064532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:49.064532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:49.064532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:49.064532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:49.064532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:49.064532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:49.064532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:49.064532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:49.064935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:49.064935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:49.064935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.064935 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:49.067394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.067394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:49.067394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:49.067394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:49.067394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:49.067394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:49.067394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:49.067394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:49.067394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:49.067394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:49.067394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:49.067394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:49.067394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:49.067394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:49.067394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:49.067394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:49.067394 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:44:49.068780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:44:49.068780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:49.068780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:49.068780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:49.068780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:49.068780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:49.068780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:49.068780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:49.068780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:49.068780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:49.068780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:49.068780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:49.068780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:49.068780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:49.068780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:49.068780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:49.068780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:49.068780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:49.068780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:49.070194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:49.070194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:49.070194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.070194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:49.135996 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:49.136706 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:49.136953 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:49.137213 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:49.137768 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:49.138162 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:49.138388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.139276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:49.139276 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:49.139276 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:49.139276 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:49.139276 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:49.139276 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:49.139276 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.139276 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:49.140812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.140812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:49.140812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:49.140812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:49.140812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:49.140812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:44:49.140812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:49.140812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:49.140812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:49.140812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:49.140812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:49.140812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:49.141425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:49.141425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:49.141425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:49.141425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:49.141425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:49.141425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:49.141425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:49.141425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:49.141425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:49.141425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:49.141425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:49.141425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:49.141425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:49.142007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.142007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:49.142007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.142007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:49.142007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:49.142007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:49.142007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:49.142007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:49.142007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:49.142007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:49.142007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:49.142007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:49.142007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:49.142007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:49.142007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:49.142706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:49.142706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:49.142706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:49.142706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:44:49.142706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:49.142706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:49.142706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:49.142706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:49.142706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:49.142706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:49.142706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:49.142706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:49.142706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:49.142706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:49.142706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:49.142706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:49.142706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:49.142706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:49.142706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:49.143511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:49.143511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:49.143511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:49.143511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:49.143511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:49.143511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:53.805949 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:53.806442 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:53.806504 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:53.807789 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:53.838627 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:53.839149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:53.840041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:53.840041 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:53.840041 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:53.840041 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:53.840041 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:53.840041 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:53.840041 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:53.840041 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:53.840785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:53.840785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:53.840785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:53.840785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:53.840785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:53.840785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:44:53.840785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:53.840785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:53.840785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:53.840785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:53.840785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:53.840785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:53.841110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:53.841110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:53.841110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:53.841110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:53.841110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:53.841110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:53.841110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:53.841110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:53.841110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:53.841110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:53.841110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:53.841110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:53.841110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:53.841466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:53.841466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:53.841527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:53.841527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:53.841527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:53.841527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:53.841527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:53.841527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:53.841527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:53.841527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:53.841527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:53.841527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:53.841527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:53.841527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:53.841527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:53.841527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:53.841527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:53.841527 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:53.841926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:44:53.841926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:53.841926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:53.841926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:53.841926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:53.841926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:53.841926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:53.841926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:53.841926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:53.841926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:53.841926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:53.841926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:53.841926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:53.841926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:53.841926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:53.841926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:53.841926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:53.841926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:53.841926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:53.842389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:53.842389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:53.842389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:54.021853 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:54.022061 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:54.022154 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:54.023322 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:54.023657 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:54.023929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.024779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:54.024779 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:54.024779 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:54.024779 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:54.024779 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:54.024779 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:54.024779 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.024779 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:54.026338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.026338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:54.026338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:54.026338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:54.026338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:54.026338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:44:54.026338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:54.026338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:54.026338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:54.026338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:54.026338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:54.026338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:54.026835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:54.026835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:54.026835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:54.026835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:54.026835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:54.026835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:54.026835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:54.026835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:54.026835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:54.026835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:54.026835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:54.026835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:54.026835 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:54.027363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.027363 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:54.028074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.028074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:54.028074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:54.028074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:54.028074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:54.028074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:54.028074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:54.028074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:54.028074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:54.028074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:54.028074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:54.028074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:54.028074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:54.028074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:54.028074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:54.028074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:54.028618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:44:54.028618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:54.028618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:54.028618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:54.028618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:54.028618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:54.028618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:54.028618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:54.028618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:54.028618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:54.028618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:54.028618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:54.028618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:54.028618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:54.028618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:54.028618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:54.028618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:54.028618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:54.028618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:54.029329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:54.029329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.029329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:54.039337 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:54.039558 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:54.039624 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:54.039824 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:44:54.040315 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:54.040589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.041549 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:54.041549 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:54.041549 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:54.041549 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:54.041549 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:54.041549 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:54.041549 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.041549 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:54.042829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.042829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:44:54.042829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:54.042829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:54.042829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:54.042829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:54.042829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:54.042829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:54.042829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:54.042829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:54.042829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:54.042829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:44:54.042829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:54.042829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:44:54.042829 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:44:54.046438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:54.046438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:44:54.046438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:54.046438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:44:54.046438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:44:54.046438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:54.046438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:44:54.046438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.046438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:44:54.046438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.046438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:54.046438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:54.046438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:54.046438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:54.047978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:44:54.047978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:44:54.047978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:44:54.047978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:54.047978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:54.047978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:54.047978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:54.047978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:54.047978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:54.047978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:54.047978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:54.047978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:54.047978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:54.048878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:54.048878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:54.048878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:54.048878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:54.048878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:54.048878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:54.048878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:54.048878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:54.048878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:54.048878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:54.048878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.048878 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:54.049535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.049535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:54.049535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:54.049535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:54.049535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:54.049535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:54.049535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:54.049535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:54.049535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:54.049535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:54.049535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:54.049535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:54.049535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:54.049535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:54.049535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:54.049535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:54.050382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:44:54.050382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:44:54.050382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:44:54.050382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:54.050382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:54.050382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:54.050382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:54.050382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:54.050382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:54.050382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:54.050382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:54.050382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:54.050382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:54.050382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:54.050382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:54.050382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:54.050382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:54.050382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:54.051221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:54.051221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:54.051221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:54.051221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:54.051221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:54.051221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.051221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:54.059966 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:54.060317 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:54.060493 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:54.060685 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:54.061263 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:54.061654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.062538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:54.062538 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:54.062538 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:54.062538 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:54.062538 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:54.062538 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:54.062538 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.062538 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:54.063812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.063812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:44:54.063812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:54.063812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:54.063812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:54.063812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:54.063812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:54.063812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:54.063812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:54.063812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:54.063812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:54.063812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:44:54.063812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:54.063812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:44:54.063812 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:44:54.064615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:54.064615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:44:54.064615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:54.064615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:44:54.064615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:44:54.064615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:54.064615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:44:54.064615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.064615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:44:54.065668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.065668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:54.065668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:54.065668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:54.065668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:54.065668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:44:54.065668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:44:54.065668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:54.065668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:54.065668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:54.065668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:54.066224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:54.066224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:54.066224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:54.066224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:54.066224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:54.066224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:54.066224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:54.066224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:54.066224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:54.066224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:54.066224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:54.066224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:54.066224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:54.066224 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:54.066979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:54.066979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:54.066979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.066979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:54.067827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.067827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:54.067827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:54.067827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:54.067827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:54.067827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:54.067827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:54.067827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:54.067827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:54.067827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:54.067827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:54.067827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:54.067827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:54.067827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:54.067827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:54.067827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:54.067827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:44:54.068555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:44:54.068555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:54.068555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:54.068555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:54.068555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:54.068555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:54.068555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:54.068555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:54.068555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:54.068555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:54.068555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:54.068555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:54.068555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:54.068555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:54.068555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:54.068555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:54.068555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:54.068555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:54.068555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:54.069497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:54.069497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:54.069497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.069497 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:54.136620 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:54.136972 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:54.137164 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:54.137442 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:54.138249 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:54.138655 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:54.138924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.139758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:54.139758 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:54.139758 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:54.139758 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:54.139758 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:54.139758 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:54.139758 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.139758 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:54.141139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.141139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:54.141139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:54.141139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:54.141139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:54.141139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:44:54.141139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:54.141139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:54.141139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:54.141139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:54.141139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:54.141139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:54.141825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:54.141825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:54.141825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:54.141825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:54.141825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:54.141825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:54.141825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:54.141825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:54.141825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:54.141825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:54.141825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:54.141825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:54.141825 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:54.142489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.142489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:54.142597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.142597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:54.142597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:54.142597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:54.142597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:54.142597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:54.142597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:54.142597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:54.142597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:54.142597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:54.142597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:54.142597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:54.142597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:54.142597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:54.142597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:54.142597 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:54.143357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:44:54.143357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:54.143357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:54.143357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:54.143357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:54.143357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:54.143357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:54.143357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:54.143357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:54.143357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:54.143357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:54.143357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:54.143357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:54.143357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:54.143357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:54.143357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:54.143357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:54.143357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:54.143357 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:54.144261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:54.144261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:54.144261 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:58.806527 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:58.807196 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:58.807196 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:58.807944 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:58.848247 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:58.848490 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:58.849061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:58.849061 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:58.849061 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:58.849061 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:58.849061 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:58.849061 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:58.849061 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:58.849061 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:58.850123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:58.850123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:58.850123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:58.850123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:58.850123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:58.850123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:44:58.850123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:58.850123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:58.850123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:58.850123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:58.850123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:58.850123 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:58.850861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:58.850861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:58.850861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:58.850861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:58.850861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:58.850861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:58.850861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:58.850861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:58.850861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:58.850861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:58.850861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:58.850861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:58.850861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:58.851576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:58.851576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:58.851576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:58.851576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:58.851576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:58.851576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:58.851576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:58.851576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:58.851576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:58.851576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:58.851576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:58.851576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:58.851576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:58.851576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:58.851576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:58.852508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:58.852508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:58.852508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:58.852508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:44:58.852508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:58.852508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:58.852508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:58.852508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:58.852508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:58.852508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:58.852508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:58.852508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:58.852508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:58.852508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:58.852508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:58.852508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:58.852508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:58.852508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:58.852508 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:58.853822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:58.853822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:58.853822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:58.853822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:58.853822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:58.853822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:59.022384 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:59.022870 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:59.023098 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:59.024144 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:59.024356 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:59.024580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.025534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:59.025534 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:59.025534 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:59.025534 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:59.025534 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:59.025534 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:59.025534 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.025534 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:59.027009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.027009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:59.027009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:59.027009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:59.027009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:59.027009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:44:59.027009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:59.027009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:59.027009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:59.027009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:59.027009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:59.027009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:59.027638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:59.027638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:59.027638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:59.027638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:59.027638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:59.027638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:59.027638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:59.027638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:59.027638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:59.027638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:59.027638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:59.027638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:59.027638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:59.028317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.028317 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:59.028766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.028766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:59.028766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:59.028766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:59.028766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:59.028766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:59.028766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:59.028766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:59.028766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:59.028766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:59.028766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:59.028766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:59.028766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:59.028766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:59.028766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:59.028766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:59.029561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:44:59.029561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:59.029561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:59.029561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:59.029561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:59.029561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:59.029561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:59.029561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:59.029561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:59.029561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:59.029561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:59.029561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:59.029561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:59.029561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:59.029561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:59.029561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:59.029561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:59.029561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:59.029561 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:59.030630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:59.030630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.030630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:59.039864 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:59.040181 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:59.040292 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:59.040365 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:44:59.040873 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:59.041226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.042074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:59.042074 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:59.042074 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:59.042074 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:59.042074 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:59.042074 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:59.042074 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.042074 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:59.042962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.042962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:44:59.042962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:59.042962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:59.042962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:59.042962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:59.042962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:59.042962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:59.042962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:59.042962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:59.042962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:59.042962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:44:59.042962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:59.042962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:44:59.042962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:44:59.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:59.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:44:59.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:59.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:44:59.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:44:59.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:59.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:44:59.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:44:59.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:59.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:59.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:59.043929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:59.044633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:44:59.044633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:44:59.044633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:44:59.044633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:59.044633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:59.044633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:59.044633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:59.044633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:59.044633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:59.044633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:59.044633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:59.044633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:59.044633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:59.045312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:59.045312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:59.045312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:59.045312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:59.045312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:59.045312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:59.045312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:59.045312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:59.045312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:59.045312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:59.045312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.045312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:59.045893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.045893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:59.045893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:59.045893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:59.045893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:59.045893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:59.045893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:59.045893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:59.045893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:59.045893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:59.045893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:59.045893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:59.045893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:59.045893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:59.045893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:59.045893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:59.046787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:44:59.046787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:44:59.046787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:44:59.046787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:59.046787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:59.046787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:59.046787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:59.046787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:59.046787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:59.046787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:59.046787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:59.046787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:59.046787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:59.046787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:59.046787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:59.046787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:59.046787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:59.046787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:59.047661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:59.047661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:59.047661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:59.047661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:59.047661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:59.047661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.047661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:59.060510 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:59.060914 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:59.061111 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:59.061295 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:59.062615 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:59.062615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.062956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:59.062956 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:59.062956 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:59.062956 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:59.062956 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:59.062956 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:59.062956 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.062956 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:59.063707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.063707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:44:59.063707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:59.063707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:59.063707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:59.063707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:59.063707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:59.063707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:59.063707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:59.063707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:44:59.063707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:59.063707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:44:59.063707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:59.063707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:44:59.063707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:44:59.065239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:59.065239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:44:59.065239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:59.065239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:44:59.065239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:44:59.065239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:59.065239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:44:59.065239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.065239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:44:59.065239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.065239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:59.065239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:59.065239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:59.065239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:59.066255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:44:59.066255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:44:59.066255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:59.066255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:59.066255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:59.066255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:59.066255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:59.066255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:59.066255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:59.066255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:59.066255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:59.066255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:59.066255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:59.067068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:59.067068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:59.067068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:59.067068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:59.067068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:59.067068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:59.067068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:59.067068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:59.067068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:59.067068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.067068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:59.067068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.067592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:59.067592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:59.067592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:59.067592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:59.067592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:59.067592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:59.067592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:59.067592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:59.067592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:59.067592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:59.067592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:59.067592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:59.067592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:59.067592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:59.067592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:59.067592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:44:59.067592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:44:59.067592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:44:59.067592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:44:59.068147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:59.068147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:59.068147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:59.068147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:59.068147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:59.068147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:59.068147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:59.068147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:59.068147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:59.068147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:59.068147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:59.068147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:59.068147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:59.068147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:59.068147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:59.068147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:59.068147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:59.068147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:59.068147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.068147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:44:59.137346 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:44:59.137820 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:44:59.137965 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:44:59.138044 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:44:59.138700 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:59.139147 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:44:59.139378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.140083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:44:59.140083 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:59.140083 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:44:59.140083 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:44:59.140083 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:59.140083 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:44:59.140083 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.140083 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:44:59.140888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.140888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:44:59.140888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:59.140888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:44:59.140888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:59.140888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:44:59.140888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:59.140888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:59.140888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:59.140888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:59.140888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:59.140888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:59.141502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:44:59.141502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:59.141502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:44:59.141502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:59.141502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:44:59.141502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:44:59.141502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:59.141502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:44:59.141502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:59.141502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:44:59.141502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:44:59.141502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:59.141502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:44:59.142228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.142228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:44:59.142228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.142228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:44:59.142228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:44:59.142228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:44:59.142228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:44:59.142228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:44:59.142228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:59.142228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:44:59.142228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:44:59.142228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:59.142228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:44:59.142228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:44:59.142228 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:44:59.143289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:44:59.143289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:44:59.143289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:44:59.143289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:44:59.143289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:44:59.143289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:59.143289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:59.143289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:44:59.143289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:44:59.143289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:44:59.143289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:44:59.143289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:44:59.143289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:44:59.143289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:44:59.143289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:44:59.143289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:44:59.143289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:44:59.143289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:44:59.143289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:44:59.144147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:44:59.144147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:44:59.144147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:44:59.144147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:44:59.144147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:44:59.144147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:03.807194 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:03.807813 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:03.807813 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:03.808467 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:03.838362 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:03.838626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:03.839471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:03.839471 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:03.839471 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:03.839471 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:03.839471 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:03.839471 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:03.839471 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:03.839471 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:03.840486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.032 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:03.840486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:03.840486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:03.840486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:03.840486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:03.840486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:45:03.840486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:03.840486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:03.840486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:03.840486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:03.840486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:03.840486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:03.840912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:03.840912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:03.840912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:03.840912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:03.840912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:03.840912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:03.840912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:03.840912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:03.840912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:03.840912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:03.840912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:03.840912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:03.840912 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:03.841328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:03.841328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:03.841524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:03.841524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:03.841524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:03.841524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:03.841524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:03.841524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:03.841524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:03.841524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:03.841524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:03.841524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:03.841524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:03.841524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:03.841524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:03.841524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:03.841524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:03.841524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:03.842067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:45:03.842067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:03.842067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:03.842067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:03.842067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:03.842067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:03.842067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:03.842067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:03.842067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:03.842067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:03.842067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:03.842067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:03.842067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:03.842067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:03.842067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:03.842067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:03.842067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:03.842067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:03.842067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:03.842748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:03.842748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:03.842748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:04.023008 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:04.023401 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:04.023594 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:04.024779 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:04.025142 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:04.025474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.026526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:04.026526 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:04.026526 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:04.026526 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:04.026526 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:04.026526 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:04.026526 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.026526 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:04.027542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.027542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:04.027542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:04.027542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:04.027542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:04.027542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:45:04.027542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:04.027542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:04.027542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:04.027542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:04.027542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:04.027542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:04.027926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:04.027926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:04.027926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:04.027926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:04.027926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:04.027926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:04.027926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:04.027926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:04.027926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:04.027926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:04.027926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:04.027926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:04.027926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:04.028275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.028275 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:04.028623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.028623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:04.028623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:04.028623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:04.028623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:04.028623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:04.028623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:04.028623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:04.028623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:04.028623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:04.028623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:04.028623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:04.028623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:04.028623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:04.028623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:04.028623 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:04.029090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:45:04.029090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:04.029090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:04.029090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:04.029090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:04.029090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:04.029090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:04.029090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:04.029090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:04.029090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:04.029090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:04.029090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:04.029090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:04.029090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:04.029090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:04.029090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:04.029090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:04.029090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:04.029090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:04.029729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:04.029729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.029729 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:04.040234 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:04.040450 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:04.040571 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:04.040830 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:45:04.041534 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:04.041987 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.042918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:04.042918 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:04.042918 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:04.042918 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:04.042918 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:04.042918 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:04.042918 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.042918 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:04.044630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.044630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:45:04.044630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:04.044630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:04.044630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:04.044630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:04.044630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:04.044630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:04.044630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:04.044630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:04.044630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:04.044630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:45:04.044630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:04.044630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:45:04.044630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:45:04.045555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:04.045555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:45:04.045555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:04.045555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:45:04.045555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:45:04.045555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:04.045555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:45:04.045555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.045555 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:45:04.046039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.046039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:04.046039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:04.046039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:04.046039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:04.046039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:45:04.046039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:45:04.046039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:45:04.046039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:04.046039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:04.046039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:04.046039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:04.046712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:04.046712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:04.046712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:04.046712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:04.046712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:04.046712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:04.046712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:04.046712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:04.046712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:04.046712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:04.046712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:04.046712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:04.046712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:04.046712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:04.047312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:04.047312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:04.047312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.047312 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:04.047867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.047867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:04.047867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:04.047867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:04.047867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:04.047867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:04.047867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:04.047867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:04.047867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:04.047867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:04.047867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:04.047867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:04.047867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:04.047867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:04.047867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:04.047867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:04.048605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:45:04.048605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:45:04.048605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:45:04.048605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:04.048605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:04.048605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:04.048605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:04.048605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:04.048605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:04.048605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:04.048605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:04.048605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:04.048605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:04.048605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:04.048605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:04.048605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:04.048605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:04.048605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:04.049725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:04.049725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:04.049725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:04.049725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:04.049725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:04.049725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.049725 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:04.060966 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:04.061205 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:04.061326 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:04.061446 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:04.062006 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:04.062421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.063226 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:04.063226 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:04.063226 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:04.063226 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:04.063226 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:04.063226 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:04.063226 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.063226 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:04.064252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.064252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:45:04.064252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:04.064252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:04.064252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:04.064252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:04.064252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:04.064252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:04.064252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:04.064252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:04.064252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:04.064252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:45:04.064252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:04.064252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:45:04.064252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:45:04.064946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:04.064946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:45:04.064946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:04.064946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:45:04.064946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:45:04.064946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:04.064946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:45:04.064946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.064946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:45:04.065843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.065843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:04.065843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:04.065843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:04.065843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:04.065843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:45:04.065843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:45:04.065843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:04.065843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:04.065843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:04.065843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:04.066436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:04.066436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:04.066436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:04.066436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:04.066436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:04.066436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:04.066436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:04.066436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:04.066436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:04.066436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:04.066436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:04.066436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:04.066436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:04.066436 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:04.067109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:04.067109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:04.067109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.067109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:04.068107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.068107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:04.068107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:04.068107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:04.068107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:04.068107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:04.068107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:04.068107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:04.068107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:04.068107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:04.068107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:04.068107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:04.068107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:04.068107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:04.068107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:04.068107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:04.068107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:45:04.068992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:45:04.068992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:04.068992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:04.068992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:04.068992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:04.068992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:04.068992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:04.068992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:04.068992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:04.068992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:04.068992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:04.068992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:04.068992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:04.068992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:04.068992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:04.068992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:04.068992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:04.068992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:04.068992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:04.069939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:04.069939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:04.069939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.069939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:04.138082 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:04.138775 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:04.138966 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:04.139256 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:04.139945 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:04.140336 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:04.140572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.141534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:04.141534 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:04.141534 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:04.141534 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:04.141534 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:04.141534 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:04.141534 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.141534 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:04.143374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.143374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:04.143374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:04.143374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:04.143374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:04.143374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:45:04.143374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:04.143374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:04.143374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:04.143374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:04.143374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:04.143374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:04.143767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:04.143767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:04.143767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:04.143767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:04.143767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:04.143767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:04.143767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:04.143767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:04.143767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:04.143767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:04.143767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:04.143767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:04.143767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:04.144103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.144103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:04.144405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.144405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:04.144405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:04.144405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:04.144405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:04.144405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:04.144405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:04.144405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:04.144405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:04.144405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:04.144405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:04.144405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:04.144405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:04.144405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:04.144405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:04.144405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:04.144862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:45:04.144862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:04.144862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:04.144862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:04.144862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:04.144862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:04.144862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:04.144862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:04.144862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:04.144862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:04.144862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:04.144862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:04.144862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:04.144862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:04.144862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:04.144862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:04.144862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:04.144862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:04.144862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:04.145379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:04.145379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:04.145379 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:08.807985 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:08.808602 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:08.809050 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:08.810756 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:08.811955 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=240 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:45:08.842933 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:08.843135 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:08.844575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:08.844575 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:08.844575 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:08.844575 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:08.844575 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:08.844575 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:08.844575 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:08.844575 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:08.844575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:08.844575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:08.844575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:08.844575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:08.844575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:08.844575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:45:08.844575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:08.845042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:08.845042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:08.845042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:08.845042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:08.845042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:08.845042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:08.845042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:08.845042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:08.845042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:08.845042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:08.845042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:08.845042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:08.845042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:08.845042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:08.845370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:08.845370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:08.845370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:08.845370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:08.845370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:08.845370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:08.845370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:08.845370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:08.845370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:08.845370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:08.845370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:08.845370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:08.845370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:08.845370 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:08.845808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:08.845808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:08.845808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:08.845808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:08.845808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:08.845808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:08.845808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:08.845808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:08.845808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:45:08.845808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:08.845808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:08.845808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:08.845808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:08.845808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:08.845808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:08.845808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:08.845808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:08.845808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:08.845808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:08.845808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:08.845808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:08.846439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:08.846439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:08.846439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:08.846439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:08.846439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:08.846439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:08.846439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:08.846439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:08.846439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:09.023732 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:09.024441 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:09.024654 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:09.026093 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:09.026569 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:09.026937 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.027946 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:09.027946 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:09.027946 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:09.027946 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:09.027946 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:09.027946 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:09.027946 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.027946 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:09.029621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.029621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:09.029621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:09.029621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:09.029621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:09.029621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:45:09.029621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:09.029621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:09.029621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:09.029621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:09.029621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:09.029621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:09.030205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:09.030205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:09.030205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:09.030205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:09.030205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:09.030205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:09.030205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:09.030205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:09.030205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:09.030205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:09.030205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:09.030205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:09.030205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:09.030780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.030780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:09.031412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.031412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:09.031412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:09.031412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:09.031412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:09.031412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:09.031412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:09.031412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:09.031412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:09.031412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:09.031412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:09.031412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:09.031412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:09.031412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:09.031412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:09.031412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:09.032221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:45:09.032221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:09.032221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:09.032221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:09.032221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:09.032221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:09.032221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:09.032221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:09.032221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:09.032221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:09.032221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:09.032221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:09.032221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:09.032221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:09.032221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:09.032221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:09.032221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:09.032221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:09.032221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:09.033218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:09.033218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.033218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:09.040581 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:09.040893 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:09.041084 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:09.041241 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:45:09.041635 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:09.041845 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.042454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:09.042454 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:09.042454 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:09.042454 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:09.042454 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:09.042454 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:09.042454 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.042454 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:09.043931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.043931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:45:09.043931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:09.043931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:09.043931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:09.043931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:09.043931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:09.043931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:09.043931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:09.043931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:09.043931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:09.043931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:45:09.043931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:09.043931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:45:09.043931 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:45:09.044699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:09.044699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:45:09.044699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:09.044699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:45:09.044699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:45:09.044699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:09.044699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:45:09.044699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.044699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:45:09.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:09.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:09.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:09.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:09.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:45:09.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:45:09.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:45:09.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:09.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:09.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:09.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:09.046246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:09.046246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:09.046246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:09.046246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:09.046246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:09.046246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:09.046246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:09.046246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:09.046246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:09.046246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:09.046246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:09.046246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:09.046246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:09.046246 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:09.046842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:09.046842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:09.046842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.046842 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:09.047265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.047265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:09.047265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:09.047265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:09.047265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:09.047265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:09.047265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:09.047265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:09.047265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:09.047265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:09.047265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:09.047265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:09.047265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:09.047265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:09.047265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:09.047265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:09.047988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:45:09.047988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:45:09.047988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:45:09.047988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:09.047988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:09.047988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:09.047988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:09.047988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:09.047988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:09.047988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:09.047988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:09.047988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:09.047988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:09.047988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:09.047988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:09.047988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:09.047988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:09.047988 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:09.049716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:09.049716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:09.049716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:09.049716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:09.049716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:09.049716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.049716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:09.061480 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:09.061864 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:09.062069 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:09.062278 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:09.062729 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:09.062955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.063605 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:09.063605 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:09.063605 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:09.063605 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:09.063605 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:09.063605 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:09.063605 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.063605 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:09.064963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.064963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:45:09.064963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:09.064963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:09.064963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:09.064963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:09.064963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:09.064963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:09.064963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:09.064963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:09.064963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:09.064963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:45:09.064963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:09.064963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:45:09.064963 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:45:09.065387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:09.065387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:45:09.065387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:09.065387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:45:09.065387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:45:09.065387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:09.065387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:45:09.065387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.065387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:45:09.066667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.066667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:09.066667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:09.066667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:09.066667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:09.066667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:45:09.066667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:45:09.066667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:09.066667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:09.066667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:09.066667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:09.067034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:09.067034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:09.067034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:09.067034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:09.067034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:09.067034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:09.067034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:09.067034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:09.067034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:09.067034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:09.067034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:09.067034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:09.067034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:09.067034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:09.067408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:09.067408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:09.067408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.067408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:09.068602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.068602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:09.068602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:09.068602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:09.068602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:09.068602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:09.068602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:09.068602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:09.068602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:09.068602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:09.068602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:09.068602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:09.068602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:09.068602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:09.068602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:09.068602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:09.068602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:45:09.069393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:45:09.069393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:09.069393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:09.069393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:09.069393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:09.069393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:09.069393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:09.069393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:09.069393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:09.069393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:09.069393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:09.069393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:09.069393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:09.069393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:09.069393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:09.069393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:09.069393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:09.069393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:09.069393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:09.070171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:09.070171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:09.070171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.070171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:09.138627 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:09.139234 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:09.139429 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:09.139774 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:09.140336 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:09.140713 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:09.140924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.141813 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:09.141813 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:09.141813 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:09.141813 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:09.141813 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:09.141813 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:09.141813 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.141813 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:09.143270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.143270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:09.143270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:09.143270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:09.143270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:09.143270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:45:09.143270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:09.143270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:09.143270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:09.143270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:09.143270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:09.143270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:09.143911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:09.143911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:09.143911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:09.143911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:09.143911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:09.143911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:09.143911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:09.143911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:09.143911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:09.143911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:09.143911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:09.143911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:09.143911 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:09.144524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.144524 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:09.144924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.144924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:09.144924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:09.144924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:09.144924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:09.144924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:09.144924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:09.144924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:09.144924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:09.144924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:09.144924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:09.144924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:09.144924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:09.144924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:09.144924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:09.144924 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:09.145750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:45:09.145750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:09.145750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:09.145750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:09.145750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:09.145750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:09.145750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:09.145750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:09.145750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:09.145750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:09.145750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:09.145750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:09.145750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:09.145750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:09.145750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:09.145750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:09.145750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:09.145750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:09.145750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:09.146647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:09.146647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:09.146647 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:13.808872 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:13.809818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:13.809818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:13.811069 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:13.849629 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:13.849629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:13.850096 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:13.850096 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:13.850096 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:13.850096 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:13.850096 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:13.850096 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:13.850096 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:13.850096 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:13.851081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:13.851081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:13.851081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:13.851081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:13.851081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:13.851081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:45:13.851081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:13.851081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:13.851081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:13.851081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:13.851081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:13.851081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:13.851515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:13.851515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:13.851515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:13.851515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:13.851515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:13.851515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:13.851515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:13.851515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:13.851515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:13.851515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:13.851515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:13.851515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:13.851515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:13.852001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:13.852001 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:13.852758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:13.852758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:13.852758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:13.852758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:13.852758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:13.852758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:13.852758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:13.852758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:13.852758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:13.852758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:13.852758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:13.852758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:13.852758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:13.852758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:13.852758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:13.852758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:13.853783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:45:13.853783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:13.853783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:13.853783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:13.853783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:13.853783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:13.853783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:13.853783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:13.853783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:13.853783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:13.853783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:13.853783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:13.853783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:13.853783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:13.853783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:13.853783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:13.853783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:13.853783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:13.853783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:13.854819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:13.854819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:13.854819 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:14.024518 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:14.025184 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:14.025397 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:14.026427 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:14.026872 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:14.027188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.028159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:14.028159 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:14.028159 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:14.028159 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:14.028159 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:14.028159 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:14.028159 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.028159 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:14.029713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.029713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:14.029713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:14.029713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:14.029713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:14.029713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:45:14.029713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:14.029713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:14.029713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:14.029713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:14.029713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:14.029713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:14.030489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:14.030489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:14.030489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:14.030489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:14.030489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:14.030489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:14.030489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:14.030489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:14.030489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:14.030489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:14.030489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:14.030489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:14.030489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:14.031039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.031039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:14.031566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.031566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:14.031566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:14.031566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:14.031566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:14.031566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:14.031566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:14.031566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:14.031566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:14.031566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:14.031566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:14.031566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:14.031566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:14.031566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:14.031566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:14.031566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:14.032338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:45:14.032338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:14.032338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:14.032338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:14.032338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:14.032338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:14.032338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:14.032338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:14.032338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:14.032338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:14.032338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:14.032338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:14.032338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:14.032338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:14.032338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:14.032338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:14.032338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:14.032338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:14.032338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:14.034507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:14.034507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.034507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:14.040862 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:14.041062 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:14.041172 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:14.041284 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:45:14.041753 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:14.042047 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.042784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:14.042784 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:14.042784 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:14.042784 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:14.042784 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:14.042784 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:14.042784 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.042784 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:14.043977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.043977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:45:14.043977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:14.043977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:14.043977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:14.043977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:14.043977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:14.043977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:14.043977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:14.043977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:14.043977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:14.043977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:45:14.043977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:14.043977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:45:14.043977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:45:14.044467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:14.044467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:45:14.044467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:14.044467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:45:14.044467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:45:14.044467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:14.044467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:45:14.044467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.044467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:45:14.045269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.045269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:14.045269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:14.045269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:14.045269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:14.045269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:45:14.045269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:45:14.045269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:45:14.045269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:14.045269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:14.045269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:14.045269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:14.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:14.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:14.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:14.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:14.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:14.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:14.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:14.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:14.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:14.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:14.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:14.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:14.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:14.045646 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:14.046164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:14.046164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:14.046164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.046164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:14.046875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.046875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:14.046875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:14.046875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:14.046875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:14.046875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:14.046875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:14.046875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:14.046875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:14.046875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:14.046875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:14.046875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:14.046875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:14.046875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:14.046875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:14.046875 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:14.047773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:45:14.047773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:45:14.047773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:45:14.047773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:14.047773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:14.047773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:14.047773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:14.047773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:14.047773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:14.047773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:14.047773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:14.047773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:14.047773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:14.047773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:14.047773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:14.047773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:14.047773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:14.047773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:14.048657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:14.048657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:14.048657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:14.048657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:14.048657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:14.048657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.048657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:14.062015 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:14.062248 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:14.062419 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:14.062622 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:14.063135 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:14.063477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.064302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:14.064302 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:14.064302 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:14.064302 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:14.064302 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:14.064302 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:14.064302 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.064302 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:14.065602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.065602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:45:14.065602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:14.065602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:14.065602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:14.065602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:14.065602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:14.065602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:14.065602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:14.065602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:14.065602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:14.065602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:45:14.065602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:14.065602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:45:14.065602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:45:14.066329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:14.066329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:45:14.066329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:14.066329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:45:14.066329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:45:14.066329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:14.066329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:45:14.066329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.066329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:45:14.067057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.067057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:14.067057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:14.067057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:14.067057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:14.067057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:45:14.067057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:45:14.067057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:14.067057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:14.067057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:14.067057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:14.067622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:14.067622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:14.067622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:14.067622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:14.067622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:14.067622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:14.067622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:14.067622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:14.067622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:14.067622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:14.067622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:14.067622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:14.067622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:14.067622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:14.068331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:14.068331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:14.068331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.068331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:14.068888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.068888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:14.068888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:14.068888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:14.068888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:14.068888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:14.068888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:14.068888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:14.068888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:14.068888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:14.068888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:14.068888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:14.068888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:14.068888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:14.068888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:14.068888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:14.068888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:45:14.069742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:45:14.069742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:14.069742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:14.069742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:14.069742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:14.069742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:14.069742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:14.069742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:14.069742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:14.069742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:14.069742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:14.069742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:14.069742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:14.069742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:14.069742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:14.069742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:14.069742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:14.069742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:14.069742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:14.070601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:14.070601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:14.070601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.070601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:14.139292 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:14.139737 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:14.139934 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:14.140193 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:14.140719 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:14.141121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:14.141346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.142164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:14.142164 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:14.142164 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:14.142164 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:14.142164 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:14.142164 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:14.142164 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.142164 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:14.143466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.143466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:14.143466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:14.143466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:14.143466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:14.143466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:45:14.143466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:14.143466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:14.143466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:14.143466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:14.143466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:14.143466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:14.144086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:14.144086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:14.144086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:14.144086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:14.144086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:14.144086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:14.144086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:14.144086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:14.144086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:14.144086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:14.144086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:14.144086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:14.144086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:14.144641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.144641 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:14.145018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.145018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:14.145018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:14.145018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:14.145018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:14.145018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:14.145018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:14.145018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:14.145018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:14.145018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:14.145018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:14.145018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:14.145018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:14.145018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:14.145018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:14.145018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:14.145768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:45:14.145768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:14.145768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:14.145768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:14.145768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:14.145768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:14.145768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:14.145768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:14.145768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:14.145768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:14.145768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:14.145768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:14.145768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:14.145768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:14.145768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:14.145768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:14.145768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:14.145768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:14.145768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:14.146577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:14.146577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:14.146577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:18.809565 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:18.810459 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:18.810878 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:18.812203 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:18.850213 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:18.850668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:18.851639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:18.851639 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:18.851639 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:18.851639 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:18.851639 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:18.851639 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:18.851639 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:18.851639 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:18.852913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.041 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:18.852913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:18.852913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:18.852913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:18.852913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:18.852913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:45:18.852913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:18.852913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:18.852913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:18.852913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:18.852913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:18.852913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:18.853936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:18.853936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:18.853936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:18.853936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:18.853936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:18.853936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:18.853936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:18.853936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:18.853936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:18.853936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:18.853936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:18.853936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:18.853936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:18.855529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:18.855529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:18.855529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:18.855529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:18.855529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:18.855529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:18.855529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:18.855529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:18.855529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:18.855529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:18.855529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:18.855529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:18.855529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:18.855529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:18.855529 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:18.857440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:18.857440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:18.857440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:18.857440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:45:18.857440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:18.857440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:18.857440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:18.857440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:18.857440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:18.857440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:18.857440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:18.857440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:18.857440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:18.857440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:18.857440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:18.857440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:18.857440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:18.857440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:18.857440 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:18.858048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:18.858048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:18.858048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:18.858048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:18.858048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:18.858048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:19.025327 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:19.025664 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:19.025894 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:19.027135 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:19.027361 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:19.027596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.028507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:19.028507 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:19.028507 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:19.028507 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:19.028507 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:19.028507 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:19.028507 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.028507 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:19.030087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.030087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:19.030087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:19.030087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:19.030087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:19.030087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:45:19.030087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:19.030087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:19.030087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:19.030087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:19.030087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:19.030087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:19.030726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:19.030726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:19.030726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:19.030726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:19.030726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:19.030726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:19.030726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:19.030726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:19.030726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:19.030726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:19.030726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:19.030726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:19.030726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:19.031249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.031249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:19.032093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.032093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:19.032093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:19.032093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:19.032093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:19.032093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:19.032093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:19.032093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:19.032093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:19.032093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:19.032093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:19.032093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:19.032093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:19.032093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:19.032093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:19.032093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:19.033796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:45:19.033796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:19.033796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:19.033796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:19.033796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:19.033796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:19.033796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:19.033796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:19.033796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:19.033796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:19.033796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:19.033796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:19.033796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:19.033796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:19.033796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:19.033796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:19.033796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:19.033796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:19.033796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:19.035098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:19.035098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.035098 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:19.041258 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:19.041585 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:19.041758 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:19.041879 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:45:19.042374 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:19.042697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.043464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:19.043464 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:19.043464 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:19.043464 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:19.043464 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:19.043464 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:19.043464 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.043464 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:19.044668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.044668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:45:19.044668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:19.044668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:19.044668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:19.044668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:19.044668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:19.044668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:19.044668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:19.044668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:19.044668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:19.044668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:45:19.044668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:19.044668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:45:19.044668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:45:19.045152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:19.045152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:45:19.045152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:19.045152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:45:19.045152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:45:19.045152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:19.045152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:45:19.045152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.045152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:45:19.045992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.045992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:19.045992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:19.045992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:19.045992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:19.045992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:45:19.045992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:45:19.045992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:45:19.045992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:19.045992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:19.045992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:19.045992 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:19.046383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:19.046383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:19.046383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:19.046383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:19.046383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:19.046383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:19.046383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:19.046383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:19.046383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:19.046383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:19.046383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:19.046383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:19.046383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:19.046383 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:19.046767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:19.046767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:19.046767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.046767 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:19.047698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.047698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:19.047698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:19.047698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:19.047698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:19.047698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:19.047698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:19.047698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:19.047698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:19.047698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:19.047698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:19.047698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:19.047698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:19.047698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:19.047698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:19.047698 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:19.048147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:45:19.048147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:45:19.048147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:45:19.048147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:19.048147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:19.048147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:19.048147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:19.048147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:19.048147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:19.048147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:19.048147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:19.048147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:19.048147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:19.048147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:19.048147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:19.048147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:19.048147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:19.048147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:19.048622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:19.048622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:19.048622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:19.048622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:19.048622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:19.048622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.048622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:19.062807 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:19.063022 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:19.063185 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:19.063307 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:19.063827 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:19.064165 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.064915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:19.064915 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:19.064915 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:19.064915 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:19.064915 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:19.064915 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:19.064915 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.064915 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:19.066230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.066230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:45:19.066230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:19.066230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:19.066230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:19.066230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:19.066230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:19.066230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:19.066230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:19.066230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:19.066230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:19.066230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:45:19.066230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:19.066230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:45:19.066230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:45:19.066922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:19.066922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:45:19.066922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:19.066922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:45:19.066922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:45:19.066922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:19.066922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:45:19.066922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.066922 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:45:19.068150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.068150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:19.068150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:19.068150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:19.068150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:19.068150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:45:19.068150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:45:19.068150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:19.068150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:19.068150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:19.068150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:19.068715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:19.068715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:19.068715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:19.068715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:19.068715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:19.068715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:19.068715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:19.068715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:19.068715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:19.068715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:19.068715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:19.068715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:19.068715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:19.068715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:19.069288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:19.069288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:19.069288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.069288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:19.070484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.070484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:19.070484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:19.070484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:19.070484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:19.070484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:19.070484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:19.070484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:19.070484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:19.070484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:19.070484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:19.070484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:19.070484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:19.070484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:19.070484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:19.070484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:19.070484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:45:19.071243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:45:19.071243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:19.071243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:19.071243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:19.071243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:19.071243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:19.071243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:19.071243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:19.071243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:19.071243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:19.071243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:19.071243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:19.071243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:19.071243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:19.071243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:19.071243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:19.071243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:19.071243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:19.071243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:19.072139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:19.072139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:19.072139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.072139 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:19.139837 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:19.140449 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:19.140644 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:19.141000 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:19.141442 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:19.141799 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:19.142016 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.142954 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:19.142954 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:19.142954 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:19.142954 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:19.142954 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:19.142954 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:19.142954 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.142954 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:19.144973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.144973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:19.144973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:19.144973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:19.144973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:19.144973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:45:19.144973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:19.144973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:19.144973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:19.144973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:19.144973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:19.144973 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:19.145572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:19.145572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:19.145572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:19.145572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:19.145572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:19.145572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:19.145572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:19.145572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:19.145572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:19.145572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:19.145572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:19.145572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:19.145572 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:19.146239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.146239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:19.146719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.146719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:19.146719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:19.146719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:19.146719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:19.146719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:19.146719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:19.146719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:19.146719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:19.146719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:19.146719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:19.146719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:19.146719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:19.146719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:19.146719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:19.146719 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:19.147402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:45:19.147402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:19.147402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:19.147402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:19.147402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:19.147402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:19.147402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:19.147402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:19.147402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:19.147402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:19.147402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:19.147402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:19.147402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:19.147402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:19.147402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:19.147402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:19.147402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:19.147402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:19.147402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:19.148414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:19.148414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:19.148414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:23.810591 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:23.811395 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:23.811395 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:23.812859 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:23.846716 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:23.847073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:23.847733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:23.847733 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:23.847733 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:23.847733 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:23.847733 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:23.847733 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:23.847733 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:23.847733 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:23.848424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:23.848424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:23.848424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:23.848424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:23.848424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:23.848424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:45:23.848424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:23.848424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:23.848424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:23.848424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:23.848424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:23.848424 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:23.849039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:23.849039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:23.849039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:23.849039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:23.849039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:23.849039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:23.849039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:23.849039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:23.849039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:23.849039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:23.849039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:23.849039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:23.849039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:23.849752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:23.849752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:23.851448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:23.851448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:23.851448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:23.851448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:23.851448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:23.851448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:23.851448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:23.851448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:23.851448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:23.851448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:23.851448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:23.851448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:23.851448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:23.851448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:23.851448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:23.851448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:23.852056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:45:23.852056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:23.852056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:23.852056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:23.852056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:23.852056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:23.852056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:23.852056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:23.852056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:23.852056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:23.852056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:23.852056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:23.852056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:23.852056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:23.852056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:23.852056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:23.852056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:23.852056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:23.852056 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:23.852836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:23.852836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:23.852836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:24.025928 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:24.026718 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:24.026938 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:24.028144 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:24.028312 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:24.028556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.029511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:24.029511 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:24.029511 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:24.029511 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:24.029511 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:24.029511 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:24.029511 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.029511 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:24.031179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.031179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:24.031179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:24.031179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:24.031179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:24.031179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:45:24.031179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:24.031179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:24.031179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:24.031179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:24.031179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:24.031179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:24.031902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:24.031902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:24.031902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:24.031902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:24.031902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:24.031902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:24.031902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:24.031902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:24.031902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:24.031902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:24.031902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:24.031902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:24.031902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:24.032479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.032479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:24.032861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.032861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:24.032861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:24.032861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:24.032861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:24.032861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:24.032861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:24.032861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:24.032861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:24.032861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:24.032861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:24.032861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:24.032861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:24.032861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:24.032861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:24.032861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:24.033614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:45:24.033614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:24.033614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:24.033614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:24.033614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:24.033614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:24.033614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:24.033614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:24.033614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:24.033614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:24.033614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:24.033614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:24.033614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:24.033614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:24.033614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:24.033614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:24.033614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:24.033614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:24.033614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:24.034934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:24.034934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.034934 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:24.041656 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:24.041996 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:24.042121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:24.042259 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:45:24.042804 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:24.043025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.043629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:24.043629 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:24.043629 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:24.043629 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:24.043629 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:24.043629 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:24.043629 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.043629 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:24.044903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.044903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:45:24.044903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:24.044903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:24.044903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:24.044903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:24.044903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:24.044903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:24.044903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:24.044903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:24.044903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:24.044903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:45:24.044903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:24.044903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:45:24.044903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:45:24.045345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:24.045345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:45:24.045345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:24.045345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:45:24.045345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:45:24.045345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:24.045345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:45:24.045345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.045345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:45:24.046515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.046515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:24.046515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:24.046515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:24.046515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:24.046515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:45:24.046515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:45:24.046515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:45:24.046515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:24.046515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:24.046515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:24.046515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:24.046914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:24.046914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:24.046914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:24.046914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:24.046914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:24.046914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:24.046914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:24.046914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:24.046914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:24.046914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:24.046914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:24.046914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:24.046914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:24.046914 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:24.047320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:24.047320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:24.047320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.047320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:24.048517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.048517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:24.048517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:24.048517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:24.048517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:24.048517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:24.048517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:24.048517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:24.048517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:24.048517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:24.048517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:24.048517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:24.048517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:24.048517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:24.048517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:24.048517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:24.049082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:45:24.049082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:45:24.049082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:45:24.049082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:24.049082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:24.049082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:24.049082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:24.049082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:24.049082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:24.049082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:24.049082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:24.049082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:24.049082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:24.049082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:24.049082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:24.049082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:24.049082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:24.049082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:24.049569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:24.049569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:24.049569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:24.049569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:24.049569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:24.049569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.049569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:24.063207 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:24.063590 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:24.063827 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:24.064033 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:24.064631 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:24.065122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.066008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:24.066008 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:24.066008 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:24.066008 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:24.066008 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:24.066008 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:24.066008 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.066008 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:24.067253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.067253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:45:24.067253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:24.067253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:24.067253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:24.067253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:24.067253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:24.067253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:24.067253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:24.067253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:24.067253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:24.067253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:45:24.067253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:24.067253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:45:24.067253 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:45:24.068018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:24.068018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:45:24.068018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:24.068018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:45:24.068018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:45:24.068018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:24.068018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:45:24.068018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.068018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:45:24.068616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.068616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:24.068616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:24.068616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:24.068616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:24.068616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:45:24.068616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:45:24.068616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:24.068616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:24.068616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:24.068616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:24.069206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:24.069206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:24.069206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:24.069206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:24.069206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:24.069206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:24.069206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:24.069206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:24.069206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:24.069206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:24.069206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:24.069206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:24.069206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:24.069206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:24.069939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:24.069939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:24.069939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.069939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:24.069939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.069939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:24.069939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:24.069939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:24.069939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:24.069939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:24.069939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:24.069939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:24.069939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:24.069939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:24.070836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:24.070836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:24.070836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:24.070836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:24.070836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:24.070836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:24.070836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:45:24.070836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:45:24.070836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:24.070836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:24.070836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:24.070836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:24.070836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:24.070836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:24.070836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:24.070836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:24.070836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:24.070836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:24.070836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:24.071772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:24.071772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:24.071772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:24.071772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:24.071772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:24.071772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:24.071772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:24.071772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:24.071772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:24.071772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.071772 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:24.140375 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:24.141093 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:24.141272 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:24.141552 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:24.142113 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:24.142624 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:24.143004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.143930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:24.143930 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:24.143930 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:24.143930 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:24.143930 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:24.143930 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:24.143930 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.143930 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:24.145376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.145376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:24.145376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:24.145376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:24.145376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:24.145376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:45:24.145376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:24.145376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:24.145376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:24.145376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:24.145376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:24.145376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:24.145981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:24.145981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:24.145981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:24.145981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:24.145981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:24.145981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:24.145981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:24.145981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:24.145981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:24.145981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:24.145981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:24.145981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:24.145981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:24.146838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.146838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:24.146838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.146838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:24.146838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:24.146838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:24.146838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:24.146838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:24.146838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:24.146838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:24.146838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:24.146838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:24.146838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:24.146838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:24.146838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:24.147439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:24.147439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:24.147439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:24.147439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:45:24.147439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:24.147439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:24.147439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:24.147439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:24.147439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:24.147439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:24.147439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:24.147439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:24.147439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:24.147439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:24.147439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:24.147439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:24.147439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:24.147439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:24.147439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:24.148053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:24.148053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:24.148053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:24.148053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:24.148053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:24.148053 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:28.811056 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:28.811972 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:28.811972 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:28.812368 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:28.850244 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:28.850566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:28.851603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:28.851603 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:28.851603 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:28.851603 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:28.851603 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:28.851603 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:28.851603 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:28.851603 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:28.852657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:28.852657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:28.852657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:28.852657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:28.852657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:28.852657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:45:28.852657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:28.852657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:28.852657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:28.852657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:28.852657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:28.852657 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:28.853255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:28.853255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:28.853255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:28.853255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:28.853255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:28.853255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:28.853255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:28.853255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:28.853255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:28.853255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:28.853255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:28.853255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:28.853255 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:28.853808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:28.853808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:28.854804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:28.854804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:28.854804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:28.854804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:28.854804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:28.854804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:28.854804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:28.854804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:28.854804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:28.854804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:28.854804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:28.854804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:28.854804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:28.854804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:28.854804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:28.854804 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:28.855534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:45:28.855534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:28.855534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:28.855534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:28.855534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:28.855534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:28.855534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:28.855534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:28.855534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:28.855534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:28.855534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:28.855534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:28.855534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:28.855534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:28.855534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:28.855534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:28.855534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:28.855534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:28.855534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:28.856137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:28.856137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:28.856137 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:29.026819 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:29.027452 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:29.027747 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:29.028860 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:29.029522 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:29.029930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.030947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:29.030947 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:29.030947 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:29.030947 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:29.030947 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:29.030947 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:29.030947 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.030947 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:29.032484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.032484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:29.032484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:29.032484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:29.032484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:29.032484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:45:29.032484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:29.032484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:29.032484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:29.032484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:29.032484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:29.032484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:29.034589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:29.034589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:29.034589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:29.034589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:29.034589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:29.034589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:29.034589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:29.034589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:29.034589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:29.034589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:29.034589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:29.034589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:29.034589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:29.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:29.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:29.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:29.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:29.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:29.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:29.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:29.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:29.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:29.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:29.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:29.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:29.035951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:29.037039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:29.037039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:29.037039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:29.037039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:45:29.037039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:29.037039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:29.037039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:29.037039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:29.037039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:29.037039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:29.037039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:29.037039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:29.037039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:29.037039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:29.037039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:29.037039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:29.037039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:29.037039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:29.037039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:29.038019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:29.038019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:29.038019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:29.038019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:29.038019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.038019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:29.042283 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:29.042592 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:29.042701 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:29.042824 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:45:29.043359 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:29.043778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.044573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:29.044573 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:29.044573 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:29.044573 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:29.044573 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:29.044573 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:29.044573 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.044573 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:29.045277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.045277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:45:29.045277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:29.045277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:29.045277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:29.045277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:29.045277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:29.045277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:29.045277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:29.045277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:29.045277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:29.045277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:45:29.045277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:29.045277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:45:29.045277 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:45:29.045781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:29.045781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:45:29.045781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:29.045781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:45:29.045781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:45:29.045781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:29.045781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:45:29.045781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.045781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:45:29.046069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.046069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:29.046069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:29.046069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:29.046069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:29.046069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:45:29.046069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:45:29.046069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:45:29.046069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:29.046069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:29.046069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:29.046069 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:29.046419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:29.046419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:29.046419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:29.046419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:29.046419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:29.046419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:29.046419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:29.046419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:29.046419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:29.046419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:29.046419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:29.046419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:29.046419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:29.046419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:29.046782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:29.046782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:29.046782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.046782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:29.047017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.047017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:29.047017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:29.047017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:29.047017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:29.047017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:29.047017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:29.047017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:29.047017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:29.047017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:29.047017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:29.047017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:29.047017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:29.047017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:29.047017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:29.047017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:29.047461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:45:29.047461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:45:29.047461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:45:29.047461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:29.047461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:29.047461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:29.047461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:29.047461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:29.047461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:29.047461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:29.047461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:29.047461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:29.047461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:29.047461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:29.047461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:29.047461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:29.047461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:29.047461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:29.048039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:29.048039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:29.048039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:29.048039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:29.048039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:29.048039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.048039 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:29.063775 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:29.064062 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:29.064256 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:29.064418 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:29.064931 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:29.065306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.066163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:29.066163 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:29.066163 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:29.066163 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:29.066163 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:29.066163 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:29.066163 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.066163 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:29.067229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.067229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:45:29.067229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:29.067229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:29.067229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:29.067229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:29.067229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:29.067229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:29.067229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:29.067229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:29.067229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:29.067229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:45:29.067229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:29.067229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:45:29.067229 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:45:29.068413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:29.068413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:45:29.068413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:29.068413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:45:29.068413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:45:29.068413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:29.068413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:45:29.068413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.068413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:45:29.068970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.068970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:29.068970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:29.068970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:29.068970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:29.068970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:45:29.068970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:45:29.068970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:29.068970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:29.068970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:29.068970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:29.069712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:29.069712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:29.069712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:29.069712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:29.069712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:29.069712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:29.069712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:29.069712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:29.069712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:29.069712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:29.069712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:29.069712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:29.069712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:29.069712 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:29.070519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:29.070519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:29.070519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.070519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:29.070519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.070519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:29.070519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:29.070519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:29.070519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:29.070519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:29.070519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:29.070519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:29.070519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:29.070519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:29.071419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:29.071419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:29.071419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:29.071419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:29.071419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:29.071419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:29.071419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:45:29.071419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:45:29.071419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:29.071419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:29.071419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:29.071419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:29.071419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:29.071419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:29.071419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:29.071419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:29.071419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:29.071419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:29.071419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:29.072550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:29.072550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:29.072550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:29.072550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:29.072550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:29.072550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:29.072550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:29.072550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:29.072550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:29.072550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.072550 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:29.141033 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:29.141423 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:29.141582 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:29.141882 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:29.142350 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:29.142784 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:29.143004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.143854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:29.143854 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:29.143854 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:29.143854 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:29.143854 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:29.143854 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:29.143854 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.143854 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:29.145092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.145092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:29.145092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:29.145092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:29.145092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:29.145092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:45:29.145092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:29.145092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:29.145092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:29.145092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:29.145092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:29.145092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:29.145617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:29.145617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:29.145617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:29.145617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:29.145617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:29.145617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:29.145617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:29.145617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:29.145617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:29.145617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:29.145617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:29.145617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:29.145617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:29.146108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.146108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:29.146588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.146588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:29.146588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:29.146588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:29.146588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:29.146588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:29.146588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:29.146588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:29.146588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:29.146588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:29.146588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:29.146588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:29.146588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:29.146588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:29.146588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:29.146588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:29.147220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:45:29.147220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:29.147220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:29.147220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:29.147220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:29.147220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:29.147220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:29.147220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:29.147220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:29.147220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:29.147220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:29.147220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:29.147220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:29.147220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:29.147220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:29.147220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:29.147220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:29.147220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:29.147220 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:29.148019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:29.148019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:29.148019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:33.811710 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:33.812306 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:33.812554 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:33.813363 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:33.845199 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:33.845557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:33.846344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:33.846344 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:33.846344 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:33.846344 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:33.846344 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:33.846344 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:33.846344 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:33.846344 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:33.847369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:33.847369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:33.847369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:33.847369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:33.847369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:33.847369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:45:33.847369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:33.847369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:33.847369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:33.847369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:33.847369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:33.847369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:33.847852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:33.847852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:33.847852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:33.847852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:33.847852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:33.847852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:33.847852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:33.847852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:33.847852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:33.847852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:33.847852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:33.847852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:33.847852 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:33.848289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:33.848289 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:33.848640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:33.848640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:33.848640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:33.848640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:33.848640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:33.848640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:33.848640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:33.848640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:33.848640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:33.848640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:33.848640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:33.848640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:33.848640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:33.848640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:33.848640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:33.848640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:33.849827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:45:33.849827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:33.849827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:33.849827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:33.849827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:33.849827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:33.849827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:33.849827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:33.849827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:33.849827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:33.849827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:33.849827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:33.849827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:33.849827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:33.849827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:33.849827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:33.849827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:33.849827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:33.849827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:33.850974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:33.850974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:33.850974 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:34.027645 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:34.028282 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:34.028479 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:34.029521 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:34.029913 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:34.030196 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.031168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:34.031168 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:34.031168 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:34.031168 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:34.031168 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:34.031168 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:34.031168 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.031168 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:34.032638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.032638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:34.032638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:34.032638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:34.032638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:34.032638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:45:34.032638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:34.032638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:34.032638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:34.032638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:34.032638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:34.032638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:34.033471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:34.033471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:34.033471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:34.033471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:34.033471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:34.033471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:34.033471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:34.033471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:34.033471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:34.033471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:34.033471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:34.033471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:34.033471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:34.034408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.034408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:34.034983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.034983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:34.034983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:34.034983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:34.034983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:34.034983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:34.034983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:34.034983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:34.034983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:34.034983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:34.034983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:34.034983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:34.034983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:34.034983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:34.034983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:34.034983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:34.036170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:45:34.036170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:34.036170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:34.036170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:34.036170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:34.036170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:34.036170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:34.036170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:34.036170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:34.036170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:34.036170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:34.036170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:34.036170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:34.036170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:34.036170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:34.036170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:34.036170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:34.036170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:34.036170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:34.037615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:34.037615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.037615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:34.042841 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:34.043159 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:34.043277 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:34.043402 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:45:34.044019 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:34.044186 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.044949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:34.044949 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:34.044949 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:34.044949 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:34.044949 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:34.044949 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:34.044949 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.044949 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:34.045699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.045699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:45:34.045699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:34.045699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:34.045699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:34.045699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:34.045699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:34.045699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:34.045699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:34.045699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:34.045699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:34.045699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:45:34.045699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:34.045699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:45:34.045699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:45:34.046533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:34.046533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:45:34.046533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:34.046533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:45:34.046533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:45:34.046533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:34.046533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:45:34.046533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.046533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:45:34.047027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.047027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:34.047027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:34.047027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:34.047027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:34.047027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:45:34.047027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:45:34.047027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:45:34.047027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:34.047027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:34.047027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:34.047027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:34.047781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:34.047781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:34.047781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:34.047781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:34.047781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:34.047781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:34.047781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:34.047781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:34.047781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:34.047781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:34.047781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:34.047781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:34.047781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:34.047781 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:34.048512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:34.048512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:34.048512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.048512 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:34.048748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.048748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:34.048748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:34.048748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:34.048748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:34.048748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:34.048748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:34.048748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:34.048748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:34.048748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:34.048748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:34.048748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:34.048748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:34.048748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:34.048748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:34.048748 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:34.049457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:45:34.049457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:45:34.049457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:45:34.049457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:34.049457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:34.049457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:34.049457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:34.049457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:34.049457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:34.049457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:34.049457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:34.049457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:34.049457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:34.049457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:34.049457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:34.049457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:34.049457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:34.049457 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:34.050318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:34.050318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:34.050318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:34.050318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:34.050318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:34.050318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.050318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:34.064177 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:34.064473 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:34.064614 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:34.064930 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:34.065374 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:34.065760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.066556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:34.066556 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:34.066556 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:34.066556 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:34.066556 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:34.066556 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:34.066556 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.066556 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:34.068023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.068023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:45:34.068023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:34.068023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:34.068023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:34.068023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:34.068023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:34.068023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:34.068023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:34.068023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:34.068023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:34.068023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:45:34.068023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:34.068023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:45:34.068023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:45:34.068795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:34.068795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:45:34.068795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:34.068795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:45:34.068795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:45:34.068795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:34.068795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:45:34.068795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.068795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:45:34.069614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.069614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:34.069614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:34.069614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:34.069614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:34.069614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:45:34.069614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:45:34.069614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:34.069614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:34.069614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:34.069614 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:34.070190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:34.070190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:34.070190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:34.070190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:34.070190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:34.070190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:34.070190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:34.070190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:34.070190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:34.070190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:34.070190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:34.070190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:34.070190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:34.070190 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:34.070806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:34.070806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:34.070806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.070806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:34.071456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.071456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:34.071456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:34.071456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:34.071456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:34.071456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:34.071456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:34.071456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:34.071456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:34.071456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:34.071456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:34.071456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:34.071456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:34.071456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:34.071456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:34.071456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:34.071456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:45:34.072216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:45:34.072216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:34.072216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:34.072216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:34.072216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:34.072216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:34.072216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:34.072216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:34.072216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:34.072216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:34.072216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:34.072216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:34.072216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:34.072216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:34.072216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:34.072216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:34.072216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:34.072216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:34.072216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:34.073019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:34.073019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:34.073019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.073019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:34.141724 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:34.142891 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:34.142891 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:34.142891 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:34.143730 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:34.143932 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:34.144092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.145029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:34.145029 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:34.145029 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:34.145029 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:34.145029 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:34.145029 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:34.145029 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.145029 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:34.146814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.146814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:34.146814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:34.146814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:34.146814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:34.146814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:45:34.146814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:34.146814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:34.146814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:34.146814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:34.146814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:34.146814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:34.147478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:34.147478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:34.147478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:34.147478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:34.147478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:34.147478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:34.147478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:34.147478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:34.147478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:34.147478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:34.147478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:34.147478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:34.147478 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:34.148048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.148048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:34.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:34.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:34.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:34.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:34.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:34.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:34.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:34.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:34.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:34.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:34.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:34.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:34.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:34.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:34.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:34.149619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:45:34.149619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:34.149619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:34.149619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:34.149619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:34.149619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:34.149619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:34.149619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:34.149619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:34.149619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:34.149619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:34.149619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:34.149619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:34.149619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:34.149619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:34.149619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:34.149619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:34.149619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:34.149619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:34.150391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:34.150391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:34.150391 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:38.812521 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:38.813379 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:38.813379 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:38.814751 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:38.848283 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:38.848690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:38.849534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:38.849534 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:38.849534 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:38.849534 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:38.849534 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:38.849534 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:38.849534 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:38.849534 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:38.850501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:38.850501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:38.850501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:38.850501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:38.850501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:38.850501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:45:38.850501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:38.850501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:38.850501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:38.850501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:38.850501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:38.850501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:38.851146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:38.851146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:38.851146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:38.851146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:38.851146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:38.851146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:38.851146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:38.851146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:38.851146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:38.851146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:38.851146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:38.851146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:38.851146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:38.851752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:38.851752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:38.851752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:38.851752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:38.851752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:38.851752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:38.851752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:38.851752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:38.851752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:38.851752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:38.851752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:38.851752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:38.851752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:38.851752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:38.851752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:38.852564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:38.852564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:38.852564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:38.852564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:45:38.852564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:38.852564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:38.852564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:38.852564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:38.852564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:38.852564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:38.852564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:38.852564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:38.852564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:38.852564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:38.852564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:38.852564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:38.852564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:38.852564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:38.852564 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:38.853450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:38.853450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:38.853450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:38.853450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:38.853450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:38.853450 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:39.028383 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:39.029103 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:39.029245 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:39.030291 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:39.030589 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:39.030848 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.031705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:39.031705 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:39.031705 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:39.031705 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:39.031705 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:39.031705 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:39.031705 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.031705 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:39.033251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.033251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:39.033251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:39.033251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:39.033251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:39.033251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:45:39.033251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:39.033251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:39.033251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:39.033251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:39.033251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:39.033251 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:39.033961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:39.033961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:39.033961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:39.033961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:39.033961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:39.033961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:39.033961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:39.033961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:39.033961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:39.033961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:39.033961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:39.033961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:39.033961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:39.034637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.034637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:39.035227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.035227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:39.035227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:39.035227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:39.035227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:39.035227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:39.035227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:39.035227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:39.035227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:39.035227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:39.035227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:39.035227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:39.035227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:39.035227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:39.035227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:39.035227 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:39.036086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:45:39.036086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:39.036086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:39.036086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:39.036086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:39.036086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:39.036086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:39.036086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:39.036086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:39.036086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:39.036086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:39.036086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:39.036086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:39.036086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:39.036086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:39.036086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:39.036086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:39.036086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:39.036086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:39.036940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:39.036940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.036940 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:39.043226 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:39.044573 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:39.044658 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:39.044815 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:45:39.045220 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:39.045489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.046154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:39.046154 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:39.046154 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:39.046154 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:39.046154 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:39.046154 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:39.046154 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.046154 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:39.046820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.046820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:45:39.046820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:39.046820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:39.046820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:39.046820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:39.046820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:39.046820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:39.046820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:39.046820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:39.046820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:39.046820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:45:39.046820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:39.046820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:45:39.046820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:45:39.047283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:39.047283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:45:39.047283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:39.047283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:45:39.047283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:45:39.047283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:39.047283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:45:39.047283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.047283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:45:39.047548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.047548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:39.047548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:39.047548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:39.047548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:39.047548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:45:39.047548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:45:39.047548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:45:39.047548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:39.047548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:39.047548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:39.047548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:39.048060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:39.048060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:39.048060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:39.048060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:39.048060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:39.048060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:39.048060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:39.048060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:39.048060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:39.048060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:39.048060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:39.048060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:39.048060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:39.048060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:39.048455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:39.048455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:39.048455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.048455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:39.048455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.048455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:39.048455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:39.048455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:39.048455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:39.048455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:39.048455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:39.048455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:39.048455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:39.048455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:39.048970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:39.048970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:39.048970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:39.048970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:39.048970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:39.048970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:39.048970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:45:39.048970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:45:39.048970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:45:39.048970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:39.048970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:39.048970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:39.048970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:39.048970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:39.048970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:39.048970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:39.048970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:39.048970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:39.048970 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:39.049624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:39.049624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:39.049624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:39.049624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:39.049624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:39.049624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:39.049624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:39.049624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:39.049624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:39.049624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:39.049624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.049624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:39.064588 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:39.065016 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:39.065197 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:39.065385 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:39.065876 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:39.066231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.067041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:39.067041 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:39.067041 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:39.067041 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:39.067041 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:39.067041 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:39.067041 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.067041 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:39.068250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.068250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:45:39.068250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:39.068250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:39.068250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:39.068250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:39.068250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:39.068250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:39.068250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:39.068250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:39.068250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:39.068250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:45:39.068250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:39.068250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:45:39.068250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:45:39.069212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:39.069212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:45:39.069212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:39.069212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:45:39.069212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:45:39.069212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:39.069212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:45:39.069212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.069212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:45:39.069621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.069621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:39.069621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:39.069621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:39.069621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:39.069621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:45:39.069621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:45:39.069621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:39.069621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:39.069621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:39.069621 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:39.070443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:39.070443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:39.070443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:39.070443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:39.070443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:39.070443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:39.070443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:39.070443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:39.070443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:39.070443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:39.070443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:39.070443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:39.070443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:39.070443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:39.071369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:39.071369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:39.071369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.071369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:39.071369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.071369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:39.071369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:39.071369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:39.071369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:39.071369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:39.071369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:39.071369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:39.071369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:39.071369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:39.072666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:39.072666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:39.072666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:39.072666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:39.072666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:39.072666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:39.072666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:45:39.072666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:45:39.072666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:39.072666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:39.072666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:39.072666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:39.072666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:39.072666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:39.072666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:39.072666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:39.072666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:39.072666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:39.072666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:39.074563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:39.074563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:39.074563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:39.074563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:39.074563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:39.074563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:39.074563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:39.074563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:39.074563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:39.074563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.074563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:39.142124 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:39.142561 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:39.142773 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:39.143012 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:39.143475 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:39.143829 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:39.144061 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.144802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:39.144802 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:39.144802 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:39.144802 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:39.144802 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:39.144802 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:39.144802 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.144802 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:39.146089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.146089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:39.146089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:39.146089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:39.146089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:39.146089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:45:39.146089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:39.146089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:39.146089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:39.146089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:39.146089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:39.146089 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:39.146755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:39.146755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:39.146755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:39.146755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:39.146755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:39.146755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:39.146755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:39.146755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:39.146755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:39.146755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:39.146755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:39.146755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:39.146755 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:39.147323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.147323 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:39.147616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.147616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:39.147616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:39.147616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:39.147616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:39.147616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:39.147616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:39.147616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:39.147616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:39.147616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:39.147616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:39.147616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:39.147616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:39.147616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:39.147616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:39.147616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:39.148581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:45:39.148581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:39.148581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:39.148581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:39.148581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:39.148581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:39.148581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:39.148581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:39.148581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:39.148581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:39.148581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:39.148581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:39.148581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:39.148581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:39.148581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:39.148581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:39.148581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:39.148581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:39.148581 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:39.149369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:39.149369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:39.149369 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:43.813179 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:43.813889 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:43.813889 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:43.815127 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:43.847585 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:43.847939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:43.848773 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:43.848773 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:43.848773 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:43.848773 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:43.848773 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:43.848773 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:43.848773 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:43.848773 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:43.849417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:43.849417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:43.849417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:43.849417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:43.849417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:43.849417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:45:43.849417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:43.849417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:43.849417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:43.849417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:43.849417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:43.849417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:43.850042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:43.850042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:43.850042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:43.850042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:43.850042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:43.850042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:43.850042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:43.850042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:43.850042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:43.850042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:43.850042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:43.850042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:43.850042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:43.850559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:43.850559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:43.850559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:43.850559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:43.850559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:43.850559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:43.850559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:43.850559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:43.850559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:43.850559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:43.850559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:43.850559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:43.850559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:43.850559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:43.850559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:43.851263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:43.851263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:43.851263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:43.851263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:45:43.851263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:43.851263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:43.851263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:43.851263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:43.851263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:43.851263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:43.851263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:43.851263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:43.851263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:43.851263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:43.851263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:43.851263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:43.851263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:43.851263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:43.851263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:43.852012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:43.852012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:43.852012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:43.852012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:43.852012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:43.852012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:44.029038 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:44.029530 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:44.029984 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:44.030764 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:44.031068 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:44.031316 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.032164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:44.032164 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:44.032164 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:44.032164 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:44.032164 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:44.032164 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:44.032164 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.032164 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:44.033697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.033697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:44.033697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:44.033697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:44.033697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:44.033697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:45:44.033697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:44.033697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:44.033697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:44.033697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:44.033697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:44.033697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:44.034302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:44.034302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:44.034302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:44.034302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:44.034302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:44.034302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:44.034302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:44.034302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:44.034302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:44.034302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:44.034302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:44.034302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:44.034302 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:44.035006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.035006 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:44.036127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.036127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:44.036127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:44.036127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:44.036127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:44.036127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:44.036127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:44.036127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:44.036127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:44.036127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:44.036127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:44.036127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:44.036127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:44.036127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:44.036127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:44.036127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:44.036903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:45:44.036903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:44.036903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:44.036903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:44.036903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:44.036903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:44.036903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:44.036903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:44.036903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:44.036903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:44.036903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:44.036903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:44.036903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:44.036903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:44.036903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:44.036903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:44.036903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:44.036903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:44.036903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:44.037418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:44.037418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.037418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:44.044688 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:44.044986 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:44.045101 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:44.045206 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:45:44.045602 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:44.045957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.046616 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:44.046616 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:44.046616 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:44.046616 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:44.046616 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:44.046616 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:44.046616 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.046616 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:44.047814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.047814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:45:44.047814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:44.047814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:44.047814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:44.047814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:44.047814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:44.047814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:44.047814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:44.047814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:44.047814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:44.047814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:45:44.047814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:44.047814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:45:44.047814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:45:44.048460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:44.048460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:45:44.048460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:44.048460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:45:44.048460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:45:44.048460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:44.048460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:45:44.048460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.048460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:45:44.049418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.049418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:44.049418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:44.049418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:44.049418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:44.049418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:45:44.049418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:45:44.049418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:45:44.049418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:44.049418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:44.049418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:44.049418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:44.049977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:44.049977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:44.049977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:44.049977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:44.049977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:44.049977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:44.049977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:44.049977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:44.049977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:44.049977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:44.049977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:44.049977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:44.049977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:44.049977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:44.050566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:44.050566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:44.050566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.050566 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:44.051775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.051775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:44.051775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:44.051775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:44.051775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:44.051775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:44.051775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:44.051775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:44.051775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:44.051775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:44.051775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:44.051775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:44.051775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:44.051775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:44.051775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:44.051775 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:44.052494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:45:44.052494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:45:44.052494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:45:44.052494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:44.052494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:44.052494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:44.052494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:44.052494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:44.052494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:44.052494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:44.052494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:44.052494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:44.052494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:44.052494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:44.052494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:44.052494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:44.052494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:44.052494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:44.053274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:44.053274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:44.053274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:44.053274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:44.053274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:44.053274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.053274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:44.065025 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:44.065354 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:44.065486 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:44.065612 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:44.065977 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:44.066279 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.066785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:44.066785 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:44.066785 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:44.066785 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:44.066785 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:44.066785 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:44.066785 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.066785 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:44.067551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.067551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:45:44.067551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:44.067551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:44.067551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:44.067551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:44.067551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:44.067551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:44.067551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:44.067551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:44.067551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:44.067551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:45:44.067551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:44.067551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:45:44.067551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:45:44.068271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:44.068271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:45:44.068271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:44.068271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:45:44.068271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:45:44.068271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:44.068271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:45:44.068271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.068271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:45:44.068651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.068651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:44.068651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:44.068651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:44.068651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:44.068651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:45:44.068651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:45:44.068651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:44.068651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:44.068651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:44.068651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:44.069164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:44.069164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:44.069164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:44.069164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:44.069164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:44.069164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:44.069164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:44.069164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:44.069164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:44.069164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:44.069164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:44.069164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:44.069164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:44.069164 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:44.069782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:44.069782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:44.069782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.069782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:44.069782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.069782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:44.069782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:44.069782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:44.069782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:44.069782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:44.069782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:44.069782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:44.069782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:44.069782 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:44.070386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:44.070386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:44.070386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:44.070386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:44.070386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:44.070386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:44.070386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:45:44.070386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:45:44.070386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:44.070386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:44.070386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:44.070386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:44.070386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:44.070386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:44.070386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:44.070386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:44.070386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:44.070386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:44.070386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:44.071271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:44.071271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:44.071271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:44.071271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:44.071271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:44.071271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:44.071271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:44.071271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:44.071271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:44.071271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.071271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:44.142729 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:44.143147 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:44.143367 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:44.143621 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:44.144102 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:44.144462 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:44.144701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.145477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:44.145477 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:44.145477 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:44.145477 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:44.145477 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:44.145477 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:44.145477 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.145477 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:44.146846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.146846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:44.146846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:44.146846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:44.146846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:44.146846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:45:44.146846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:44.146846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:44.146846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:44.146846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:44.146846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:44.146846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:44.147420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:44.147420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:44.147420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:44.147420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:44.147420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:44.147420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:44.147420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:44.147420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:44.147420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:44.147420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:44.147420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:44.147420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:44.147420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:44.147967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.147967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:44.148449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.148449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:44.148449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:44.148449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:44.148449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:44.148449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:44.148449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:44.148449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:44.148449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:44.148449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:44.148449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:44.148449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:44.148449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:44.148449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:44.148449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:44.148449 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:44.149189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:45:44.149189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:44.149189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:44.149189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:44.149189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:44.149189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:44.149189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:44.149189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:44.149189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:44.149189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:44.149189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:44.149189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:44.149189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:44.149189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:44.149189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:44.149189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:44.149189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:44.149189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:44.149189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:44.150065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:44.150065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:44.150065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:48.813807 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:48.814531 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:48.814727 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:48.815884 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:48.847573 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:48.848152 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:48.849318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:48.849318 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:48.849318 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:48.849318 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:48.849318 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:48.849318 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:48.849318 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:48.849318 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:48.850106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.035 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:48.850106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:48.850106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:48.850106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:48.850106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:48.850106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:45:48.850106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:48.850106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:48.850106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:48.850106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:48.850106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:48.850106 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:48.850947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:48.850947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:48.850947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:48.850947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:48.850947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:48.850947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:48.850947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:48.850947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:48.850947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:48.850947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:48.850947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:48.850947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:48.850947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:48.851785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:48.851785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:48.851785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:48.851785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:48.851785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:48.851785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:48.851785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:48.851785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:48.851785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:48.851785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:48.851785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:48.851785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:48.851785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:48.851785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:48.851785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:48.852868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:48.852868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:48.852868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:48.852868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:45:48.852868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:48.852868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:48.852868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:48.852868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:48.852868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:48.852868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:48.852868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:48.852868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:48.852868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:48.852868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:48.852868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:48.852868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:48.852868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:48.852868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:48.852868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:48.853604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:48.853604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:48.853604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:48.853604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:48.853604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:48.853604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:49.029788 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:49.030314 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:49.030500 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:49.031473 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:49.031841 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:49.032086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.033037 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:49.033037 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:49.033037 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:49.033037 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:49.033037 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:49.033037 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:49.033037 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.033037 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:49.034568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.034568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:49.034568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:49.034568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:49.034568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:49.034568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:45:49.034568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:49.034568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:49.034568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:49.034568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:49.034568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:49.034568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:49.035174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:49.035174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:49.035174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:49.035174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:49.035174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:49.035174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:49.035174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:49.035174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:49.035174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:49.035174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:49.035174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:49.035174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:49.035174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:49.035770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.035770 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:49.036248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.036248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:49.036248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:49.036248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:49.036248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:49.036248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:49.036248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:49.036248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:49.036248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:49.036248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:49.036248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:49.036248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:49.036248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:49.036248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:49.036248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:49.036248 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:49.036994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:45:49.036994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:49.036994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:49.036994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:49.036994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:49.036994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:49.036994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:49.036994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:49.036994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:49.036994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:49.036994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:49.036994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:49.036994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:49.036994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:49.036994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:49.036994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:49.036994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:49.036994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:49.036994 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:49.037798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:49.037798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.037798 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:49.045139 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:49.045430 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:49.045545 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:49.045662 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:45:49.046108 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:49.046425 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.047159 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:49.047159 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:49.047159 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:49.047159 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:49.047159 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:49.047159 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:49.047159 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.047159 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:49.047938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.047938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:45:49.047938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:49.047938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:49.047938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:49.047938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:49.047938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:49.047938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:49.047938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:49.047938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:49.047938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:49.047938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:45:49.047938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:49.047938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:45:49.047938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:45:49.048403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:49.048403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:45:49.048403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:49.048403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:45:49.048403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:45:49.048403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:49.048403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:45:49.048403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.048403 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:45:49.049206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.049206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:49.049206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:49.049206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:49.049206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:49.049206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:45:49.049206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:45:49.049206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:45:49.049206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:49.049206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:49.049206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:49.049206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:49.049596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:49.049596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:49.049596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:49.049596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:49.049596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:49.049596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:49.049596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:49.049596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:49.049596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:49.049596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:49.049596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:49.049596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:49.049596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:49.049596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:49.049978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:49.049978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:49.049978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.049978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:49.051038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.051038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:49.051038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:49.051038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:49.051038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:49.051038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:49.051038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:49.051038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:49.051038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:49.051038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:49.051038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:49.051038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:49.051038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:49.051038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:49.051038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:49.051038 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:49.051477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:45:49.051477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:45:49.051477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:45:49.051477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:49.051477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:49.051477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:49.051477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:49.051477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:49.051477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:49.051477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:49.051477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:49.051477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:49.051477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:49.051477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:49.051477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:49.051477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:49.051477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:49.051477 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:49.051965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:49.051965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:49.051965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:49.051965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:49.051965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:49.051965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.051965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:49.065454 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:49.065697 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:49.065828 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:49.066098 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:49.066343 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:49.066687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.067429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:49.067429 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:49.067429 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:49.067429 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:49.067429 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:49.067429 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:49.067429 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.067429 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:49.068663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.068663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:45:49.068663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:49.068663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:49.068663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:49.068663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:49.068663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:49.068663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:49.068663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:49.068663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:49.068663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:49.068663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:45:49.068663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:49.068663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:45:49.068663 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:45:49.069099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:49.069099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:45:49.069099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:49.069099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:45:49.069099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:45:49.069099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:49.069099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:45:49.069099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.069099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:45:49.070125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.070125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:49.070125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:49.070125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:49.070125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:49.070125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:45:49.070125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:45:49.070125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:49.070125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:49.070125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:49.070125 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:49.070462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:49.070462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:49.070462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:49.070462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:49.070462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:49.070462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:49.070462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:49.070462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:49.070462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:49.070462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:49.070462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:49.070462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:49.070462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:49.070462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:49.070950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:49.070950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:49.070950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.070950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:49.071822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.071822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:49.071822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:49.071822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:49.071822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:49.071822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:49.071822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:49.071822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:49.071822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:49.071822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:49.071822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:49.071822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:49.071822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:49.071822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:49.071822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:49.071822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:49.071822 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:45:49.072267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:45:49.072267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:49.072267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:49.072267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:49.072267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:49.072267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:49.072267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:49.072267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:49.072267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:49.072267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:49.072267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:49.072267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:49.072267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:49.072267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:49.072267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:49.072267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:49.072267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:49.072267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:49.072267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:49.072810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:49.072810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:49.072810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.072810 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:49.143381 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:49.143849 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:49.144034 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:49.144272 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:49.144842 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:49.145335 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:49.145576 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.146462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:49.146462 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:49.146462 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:49.146462 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:49.146462 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:49.146462 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:49.146462 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.146462 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:49.147950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.147950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:49.147950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:49.147950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:49.147950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:49.147950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:45:49.147950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:49.147950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:49.147950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:49.147950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:49.147950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:49.147950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:49.148506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:49.148506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:49.148506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:49.148506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:49.148506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:49.148506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:49.148506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:49.148506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:49.148506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:49.148506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:49.148506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:49.148506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:49.148506 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:49.149119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.149119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:49.149759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.149759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:49.149759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:49.149759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:49.149759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:49.149759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:49.149759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:49.149759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:49.149759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:49.149759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:49.149759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:49.149759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:49.149759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:49.149759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:49.149759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:49.149759 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:49.150471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:45:49.150471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:49.150471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:49.150471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:49.150471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:49.150471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:49.150471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:49.150471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:49.150471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:49.150471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:49.150471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:49.150471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:49.150471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:49.150471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:49.150471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:49.150471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:49.150471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:49.150471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:49.150471 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:49.151467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:49.151467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:49.151467 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:53.814592 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:53.815649 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:53.815649 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:53.816794 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:53.847796 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:53.848280 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:53.849328 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:53.849328 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:53.849328 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:53.849328 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:53.849328 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:53.849328 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:53.849328 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:53.849328 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:53.850313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:53.850313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:53.850313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:53.850313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:53.850313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:53.850313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:45:53.850313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:53.850313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:53.850313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:53.850313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:53.850313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:53.850313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:53.851019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:53.851019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:53.851019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:53.851019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:53.851019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:53.851019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:53.851019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:53.851019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:53.851019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:53.851019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:53.851019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:53.851019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:53.851019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:53.851734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:53.851734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:53.851734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:53.851734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:53.851734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:53.851734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:53.851734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:53.851734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:53.851734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:53.851734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:53.851734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:53.851734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:53.851734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:53.851734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:53.851734 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:53.852560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:53.852560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:53.852560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:53.852560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:45:53.852560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:53.852560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:53.852560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:53.852560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:53.852560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:53.852560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:53.852560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:53.852560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:53.852560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:53.852560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:53.852560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:53.852560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:53.852560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:53.852560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:53.852560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:53.853562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:53.853562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:53.853562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:53.853562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:53.853562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:53.853562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:54.030668 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:54.031095 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:54.031250 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:54.032286 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:54.033290 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:54.034055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.035191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:54.035191 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:54.035191 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:54.035191 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:54.035191 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:54.035191 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:54.035191 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.035191 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:54.036864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.036864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:54.036864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:54.036864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:54.036864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:54.036864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:45:54.036864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:54.036864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:54.036864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:54.036864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:54.036864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:54.036864 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:54.037513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:54.037513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:54.037513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:54.037513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:54.037513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:54.037513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:54.037513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:54.037513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:54.037513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:54.037513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:54.037513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:54.037513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:54.037513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:54.038140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.038140 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:54.038622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.038622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:54.038622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:54.038622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:54.038622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:54.038622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:54.038622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:54.038622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:54.038622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:54.038622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:54.038622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:54.038622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:54.038622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:54.038622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:54.038622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:54.038622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:54.039376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:45:54.039376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:54.039376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:54.039376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:54.039376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:54.039376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:54.039376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:54.039376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:54.039376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:54.039376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:54.039376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:54.039376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:54.039376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:54.039376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:54.039376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:54.039376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:54.039376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:54.039376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:54.039376 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:54.039966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:54.039966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.039966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:54.045527 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:54.045750 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:54.045860 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:54.045975 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:45:54.046854 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:54.046854 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.046972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:54.046972 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:54.046972 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:54.046972 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:54.046972 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:54.046972 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:54.046972 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.046972 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:54.047699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.047699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:45:54.047699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:54.047699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:54.047699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:54.047699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:54.047699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:54.047699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:54.047699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:54.047699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:54.047699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:54.047699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:45:54.047699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:54.047699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:45:54.047699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:45:54.048214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:54.048214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:45:54.048214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:54.048214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:45:54.048214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:45:54.048214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:54.048214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:45:54.048214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.048214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:45:54.048976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.048976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:54.048976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:54.048976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:54.048976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:54.048976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:45:54.048976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:45:54.048976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:45:54.048976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:54.048976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:54.048976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:54.048976 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:54.050175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:54.050175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:54.050175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:54.050175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:54.050175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:54.050175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:54.050175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:54.050175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:54.050175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:54.050175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:54.050175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:54.050175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:54.050175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:54.050175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:54.050651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:54.050651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:54.050651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.050651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:54.050651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.050651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:54.050651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:54.050651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:54.050651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:54.050651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:54.050651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:54.050651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:54.050651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:54.050651 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:54.051073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:54.051073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:54.051073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:54.051073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:54.051073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:54.051073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:54.051073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:45:54.051073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:45:54.051073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:45:54.051073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:54.051073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:54.051073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:54.051073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:54.051073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:54.051073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:54.051073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:54.051073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:54.051073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:54.051073 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:54.051600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:54.051600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:54.051600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:54.051600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:54.051600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:54.051600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:54.051600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:54.051600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:54.051600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:54.051600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:54.051600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.051600 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:54.065884 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:54.066208 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:54.066347 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:54.066578 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:54.067063 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:54.067404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.068189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:54.068189 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:54.068189 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:54.068189 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:54.068189 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:54.068189 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:54.068189 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.068189 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:54.069413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.069413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:45:54.069413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:54.069413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:54.069413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:54.069413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:54.069413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:54.069413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:54.069413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:54.069413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:54.069413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:54.069413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:45:54.069413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:54.069413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:45:54.069413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:45:54.070141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:54.070141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:45:54.070141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:54.070141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:45:54.070141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:45:54.070141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:54.070141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:45:54.070141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.070141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:45:54.070900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.070900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:54.070900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:54.070900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:54.070900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:54.070900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:45:54.070900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:45:54.070900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:54.070900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:54.070900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:54.070900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:54.071492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:54.071492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:54.071492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:54.071492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:54.071492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:54.071492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:54.071492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:54.071492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:54.071492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:54.071492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:54.071492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:54.071492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:54.071492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:54.071492 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:54.072160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:54.072160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:54.072160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.072160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:54.072528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.072528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:54.072528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:54.072528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:54.072528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:54.072528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:54.072528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:54.072528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:54.072528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:54.072528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:54.072528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:54.072528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:54.072528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:54.072528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:54.072528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:54.072528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:54.072528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:45:54.073416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:45:54.073416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:54.073416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:54.073416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:54.073416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:54.073416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:54.073416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:54.073416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:54.073416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:54.073416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:54.073416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:54.073416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:54.073416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:54.073416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:54.073416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:54.073416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:54.073416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:54.073416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:54.073416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:54.074460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:54.074460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:54.074460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.074460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:54.143981 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:54.144471 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:54.144606 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:54.144903 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:54.145337 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:54.145712 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:54.145930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.146707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:54.146707 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:54.146707 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:54.146707 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:54.146707 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:54.146707 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:54.146707 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.146707 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:54.148276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.148276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:54.148276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:54.148276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:54.148276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:54.148276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:45:54.148276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:54.148276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:54.148276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:54.148276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:54.148276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:54.148276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:54.149218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:54.149218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:54.149218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:54.149218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:54.149218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:54.149218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:54.149218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:54.149218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:54.149218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:54.149218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:54.149218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:54.149218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:54.149218 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:54.149859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.149859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:54.149859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.149859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:54.149859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:54.149859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:54.149859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:54.149859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:54.149859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:54.149859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:54.149859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:54.149859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:54.149859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:54.149859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:54.149859 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:54.150667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:54.150667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:54.150667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:54.150667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:45:54.150667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:54.150667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:54.150667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:54.150667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:54.150667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:54.150667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:54.150667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:54.150667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:54.150667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:54.150667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:54.150667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:54.150667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:54.150667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:54.150667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:54.150667 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:54.151435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:54.151435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:54.151435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:54.151435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:54.151435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:54.151435 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:58.815282 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:58.816120 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:58.816120 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:58.817245 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:58.852107 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:58.852409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:58.853086 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:58.853086 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:58.853086 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:58.853086 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:58.853086 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:58.853086 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:58.853086 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:58.853086 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:58.853838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:58.853838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:58.853838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:58.853838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:58.853838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:58.853838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:45:58.853838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:58.853838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:58.853838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:58.853838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:58.853838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:58.853838 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:58.854469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:58.854469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:58.854469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:58.854469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:58.854469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:58.854469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:58.854469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:58.854469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:58.854469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:58.854469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:58.854469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:58.854469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:58.854469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:58.855149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:58.855149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:58.855149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:58.855149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:58.855149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:58.855149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:58.855149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:58.855149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:58.855149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:58.855149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:58.855149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:58.855149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:58.855149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:58.855149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:58.855149 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:58.855962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:58.855962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:58.855962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:58.855962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:45:58.855962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:58.855962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:58.855962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:58.855962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:58.855962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:58.855962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:58.855962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:58.855962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:58.855962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:58.855962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:58.855962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:58.855962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:58.855962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:58.855962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:58.855962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:58.856951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:58.856951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:58.856951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:58.856951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:58.856951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:58.856951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:59.031174 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:59.031608 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:59.031925 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:59.032960 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:59.033186 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:59.033320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.033983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:59.033983 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:59.033983 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:59.033983 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:59.033983 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:59.033983 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:59.033983 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.033983 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:59.035269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.035269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:59.035269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:59.035269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:59.035269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:59.035269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:45:59.035269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:59.035269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:59.035269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:59.035269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:59.035269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:59.035269 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:59.035836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:59.035836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:59.035836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:59.035836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:59.035836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:59.035836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:59.035836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:59.035836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:59.035836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:59.035836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:59.035836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:59.035836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:59.035836 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:59.036456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.036456 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:59.037778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.037778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:59.037778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:59.037778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:59.037778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:59.037778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:59.037778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:59.037778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:59.037778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:59.037778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:59.037778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:59.037778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:59.037778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:59.037778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:59.037778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:59.037778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:59.038309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:45:59.038309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:59.038309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:59.038309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:59.038309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:59.038309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:59.038309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:59.038309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:59.038309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:59.038309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:59.038309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:59.038309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:59.038309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:59.038309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:59.038309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:59.038309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:59.038309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:59.038309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:59.038309 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:59.038855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:59.038855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.038855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:59.045864 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:59.046160 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:59.046314 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:59.046484 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:45:59.046912 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:59.047183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.047876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:59.047876 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:59.047876 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:59.047876 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:59.047876 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:59.047876 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:59.047876 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.047876 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:59.048844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.048844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:45:59.048844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:59.048844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:59.048844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:59.048844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:59.048844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:59.048844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:59.048844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:59.048844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:59.048844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:59.048844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:45:59.048844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:59.048844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:45:59.048844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:45:59.049601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:59.049601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:45:59.049601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:59.049601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:45:59.049601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:45:59.049601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:59.049601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:45:59.049601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.049601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:45:59.050134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.050134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:59.050134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:59.050134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:59.050134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:59.050134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:45:59.050134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:45:59.050134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:45:59.050134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:59.050134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:59.050134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:59.050134 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:59.050718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:59.050718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:59.050718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:59.050718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:59.050718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:59.050718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:59.050718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:59.050718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:59.050718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:59.050718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:59.050718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:59.050718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:59.050718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:59.050718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:59.051390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:59.051390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:59.051390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.051390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:59.051390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.051390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:59.051390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:59.051390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:59.051390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:59.051390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:59.051390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:59.051390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:59.051390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:59.051390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:59.051881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:59.051881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:59.051881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:59.051881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:59.051881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:59.051881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:59.051881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:45:59.051881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:45:59.051881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:45:59.051881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:59.051881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:59.051881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:59.051881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:59.051881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:59.051881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:59.051881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:59.051881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:59.051881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:59.051881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:59.052507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:59.052507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:59.052507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:59.052507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:59.052507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:59.052507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:59.052507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:59.052507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:59.052507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:59.052507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:59.052507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.052507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:59.066287 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:59.066632 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:59.066914 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:59.067192 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:59.067628 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:59.068004 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.068820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:59.068820 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:59.068820 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:59.068820 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:59.068820 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:59.068820 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:59.068820 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.068820 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:59.070185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.070185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:45:59.070185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:59.070185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:59.070185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:59.070185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:59.070185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:59.070185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:59.070185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:59.070185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:45:59.070185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:59.070185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:45:59.070185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:59.070185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:45:59.070185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:45:59.071179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:59.071179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:45:59.071179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:59.071179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:45:59.071179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:45:59.071179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:59.071179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:45:59.071179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.071179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:45:59.072158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.072158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:59.072158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:59.072158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:59.072158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:59.072158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:45:59.072158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:45:59.072158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:59.072158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:59.072158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:59.072158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:59.072981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:59.072981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:59.072981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:59.072981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:59.072981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:59.072981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:59.072981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:59.072981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:59.072981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:59.072981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:59.072981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:59.072981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:59.072981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:59.072981 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:59.073899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:59.073899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:59.073899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.073899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:59.074631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.074631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:59.074631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:59.074631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:59.074631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:59.074631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:59.074631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:59.074631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:59.074631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:59.074631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:59.074631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:59.074631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:59.074631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:59.074631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:59.074631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:59.074631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:59.074631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:45:59.075469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:45:59.075469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:45:59.075469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:45:59.075469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:59.075469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:59.075469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:59.075469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:59.075469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:59.075469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:59.075469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:59.075469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:59.075469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:59.075469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:59.075469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:59.075469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:59.075469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:59.075469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:59.075469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:59.075469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:59.076017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:59.076017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:59.076017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.076017 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:45:59.144330 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:45:59.144643 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:45:59.144852 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:45:59.145065 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:45:59.145353 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:59.145580 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:45:59.145735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.146375 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:45:59.146375 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:59.146375 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:45:59.146375 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:45:59.146375 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:59.146375 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:45:59.146375 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.146375 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:45:59.147960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.147960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:45:59.147960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:59.147960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:45:59.147960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:59.147960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:45:59.147960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:59.147960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:59.147960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:59.147960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:59.147960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:59.147960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:59.148900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:45:59.148900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:59.148900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:45:59.148900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:59.148900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:45:59.148900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:45:59.148900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:59.148900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:45:59.148900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:59.148900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:45:59.148900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:45:59.148900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:59.148900 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:45:59.149899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.149899 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:45:59.150304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.150304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:45:59.150304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:45:59.150304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:45:59.150304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:45:59.150304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:45:59.150304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:59.150304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:45:59.150304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:45:59.150304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:59.150304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:45:59.150304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:45:59.150304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:45:59.150304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:45:59.150304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:45:59.150304 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:45:59.151155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:45:59.151155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:45:59.151155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:59.151155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:59.151155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:45:59.151155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:45:59.151155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:45:59.151155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:45:59.151155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:45:59.151155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:45:59.151155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:45:59.151155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:45:59.151155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:45:59.151155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:45:59.151155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:45:59.151155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:45:59.151155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:45:59.151155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:45:59.151155 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:45:59.152048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:45:59.152048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:45:59.152048 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:03.815947 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:03.816623 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:03.816623 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:03.817524 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:03.854575 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:03.855042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:03.855950 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:03.855950 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:03.855950 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:03.855950 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:03.855950 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:03.855950 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:03.855950 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:03.855950 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:03.856999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.039 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:03.856999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:03.856999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:03.856999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:03.856999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:03.856999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:46:03.856999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:03.856999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:03.856999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:03.856999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:03.856999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:03.856999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:03.857627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:03.857627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:03.857627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:03.857627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:03.857627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:03.857627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:03.857627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:03.857627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:03.857627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:03.857627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:03.857627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:03.857627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:03.857627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:03.858241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:03.858241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:03.858241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:03.858241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:03.858241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:03.858241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:03.858241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:03.858241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:03.858241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:03.858241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:03.858241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:03.858241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:03.858241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:03.858241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:03.858241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:03.859080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:03.859080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:03.859080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:03.859080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:46:03.859080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:03.859080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:03.859080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:03.859080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:03.859080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:03.859080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:03.859080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:03.859080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:03.859080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:03.859080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:03.859080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:03.859080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:03.859080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:03.859080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:03.859080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:03.860024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:03.860024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:03.860024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:03.860024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:03.860024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:03.860024 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:04.031773 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:04.032177 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:04.032292 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:04.033171 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:04.033488 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:04.033749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.034537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:04.034537 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:04.034537 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:04.034537 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:04.034537 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:04.034537 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:04.034537 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.034537 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:04.035584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.035584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:04.035584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:04.035584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:04.035584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:04.035584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:46:04.035584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:04.035584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:04.035584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:04.035584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:04.035584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:04.035584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:04.036199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:04.036199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:04.036199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:04.036199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:04.036199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:04.036199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:04.036199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:04.036199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:04.036199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:04.036199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:04.036199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:04.036199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:04.036199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:04.036774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.036774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:04.036774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.036774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:04.036774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:04.036774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:04.036774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:04.036774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:04.036774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:04.036774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:04.036774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:04.036774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:04.036774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:04.036774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:04.036774 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:04.037486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:04.037486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:04.037486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:04.037486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:46:04.037486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:04.037486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:04.037486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:04.037486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:04.037486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:04.037486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:04.037486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:04.037486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:04.037486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:04.037486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:04.037486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:04.037486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:04.037486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:04.037486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:04.037486 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:04.038314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:04.038314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:04.038314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:04.038314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:04.038314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.038314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:04.046187 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:04.046442 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:04.046556 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:04.046693 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:46:04.047098 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:04.047365 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.048035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:04.048035 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:04.048035 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:04.048035 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:04.048035 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:04.048035 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:04.048035 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.048035 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:04.048787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.048787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:46:04.048787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:04.048787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:04.048787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:04.048787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:04.048787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:04.048787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:04.048787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:04.048787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:04.048787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:04.048787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:46:04.048787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:04.048787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:46:04.048787 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:46:04.049522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:04.049522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:46:04.049522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:04.049522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:46:04.049522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:46:04.049522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:04.049522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:46:04.049522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.049522 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:46:04.050000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.050000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:04.050000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:04.050000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:04.050000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:04.050000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:46:04.050000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:46:04.050000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:46:04.050000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:04.050000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:04.050000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:04.050000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:04.050588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:04.050588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:04.050588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:04.050588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:04.050588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:04.050588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:04.050588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:04.050588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:04.050588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:04.050588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:04.050588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:04.050588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:04.050588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:04.050588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:04.051209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:04.051209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:04.051209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.051209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:04.051209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.051209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:04.051209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:04.051209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:04.051209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:04.051209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:04.051209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:04.051209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:04.051209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:04.051209 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:04.051861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:04.051861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:04.051861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:04.051861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:04.051861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:04.051861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:04.051861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:46:04.051861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:46:04.051861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:46:04.051861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:04.051861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:04.051861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:04.051861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:04.051861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:04.051861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:04.051861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:04.051861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:04.051861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:04.051861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:04.052740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:04.052740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:04.052740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:04.052740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:04.052740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:04.052740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:04.052740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:04.052740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:04.052740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:04.052740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:04.052740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.052740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:04.066734 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:04.067011 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:04.067117 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:04.067242 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:04.067666 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:04.068012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.068668 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:04.068668 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:04.068668 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:04.068668 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:04.068668 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:04.068668 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:04.068668 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.068668 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:04.069420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.069420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:46:04.069420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:04.069420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:04.069420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:04.069420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:04.069420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:04.069420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:04.069420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:04.069420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:04.069420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:04.069420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:46:04.069420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:04.069420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:46:04.069420 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:46:04.070162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:04.070162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:46:04.070162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:04.070162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:46:04.070162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:46:04.070162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:04.070162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:46:04.070162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.070162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:46:04.070583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.070583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:04.070583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:04.070583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:04.070583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:04.070583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:46:04.070583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:46:04.070583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:04.070583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:04.070583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:04.070583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:04.071114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:04.071114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:04.071114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:04.071114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:04.071114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:04.071114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:04.071114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:04.071114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:04.071114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:04.071114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:04.071114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:04.071114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:04.071114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:04.071114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:04.071733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:04.071733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:04.071733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.071733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:04.071733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.071733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:04.071733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:04.071733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:04.071733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:04.071733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:04.071733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:04.071733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:04.071733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:04.071733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:04.072353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:04.072353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:04.072353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:04.072353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:04.072353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:04.072353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:04.072353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:46:04.072353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:46:04.072353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:04.072353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:04.072353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:04.072353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:04.072353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:04.072353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:04.072353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:04.072353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:04.072353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:04.072353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:04.072353 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:04.073175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:04.073175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:04.073175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:04.073175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:04.073175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:04.073175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:04.073175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:04.073175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:04.073175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:04.073175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.073175 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:04.144880 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:04.145265 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:04.145445 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:04.145666 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:04.146200 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:04.146591 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:04.146846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.147626 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:04.147626 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:04.147626 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:04.147626 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:04.147626 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:04.147626 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:04.147626 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.147626 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:04.148953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.148953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:04.148953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:04.148953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:04.148953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:04.148953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:46:04.148953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:04.148953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:04.148953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:04.148953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:04.148953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:04.148953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:04.149530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:04.149530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:04.149530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:04.149530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:04.149530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:04.149530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:04.149530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:04.149530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:04.149530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:04.149530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:04.149530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:04.149530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:04.149530 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:04.150114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.150114 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:04.150464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.150464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:04.150464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:04.150464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:04.150464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:04.150464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:04.150464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:04.150464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:04.150464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:04.150464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:04.150464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:04.150464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:04.150464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:04.150464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:04.150464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:04.150464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:04.151249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:46:04.151249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:04.151249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:04.151249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:04.151249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:04.151249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:04.151249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:04.151249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:04.151249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:04.151249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:04.151249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:04.151249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:04.151249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:04.151249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:04.151249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:04.151249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:04.151249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:04.151249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:04.151249 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:04.152080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:04.152080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:04.152080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:08.816658 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:08.817354 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:08.817354 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:08.818490 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:08.820039 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=240 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:46:08.855898 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:08.856129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:08.856869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:08.856869 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:08.856869 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:08.856869 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:08.856869 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:08.856869 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:08.856869 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:08.856869 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:08.858193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.040 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:08.858193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:08.858193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:08.858193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:08.858193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:08.858193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:46:08.858193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:08.858193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:08.858193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:08.858193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:08.858193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:08.858193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:08.859078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:08.859078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:08.859078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:08.859078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:08.859078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:08.859078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:08.859078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:08.859078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:08.859078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:08.859078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:08.859078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:08.859078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:08.859078 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:08.859705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:08.859705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:08.859948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:08.859948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:08.859948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:08.859948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:08.859948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:08.859948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:08.859948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:08.859948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:08.859948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:08.859948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:08.859948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:08.859948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:08.859948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:08.859948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:08.859948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:08.859948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:08.860806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:46:08.860806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:08.860806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:08.860806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:08.860806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:08.860806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:08.860806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:08.860806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:08.860806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:08.860806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:08.860806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:08.860806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:08.860806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:08.860806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:08.860806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:08.860806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:08.860806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:08.860806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:08.860806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:08.861943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:08.861943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:08.861943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:09.032419 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:09.033143 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:09.033420 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:09.034494 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:09.034868 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:09.035129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.036065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:09.036065 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:09.036065 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:09.036065 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:09.036065 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:09.036065 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:09.036065 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.036065 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:09.037806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.037806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:09.037806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:09.037806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:09.037806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:09.037806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:46:09.037806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:09.037806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:09.037806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:09.037806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:09.037806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:09.037806 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:09.038402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:09.038402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:09.038402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:09.038402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:09.038402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:09.038402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:09.038402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:09.038402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:09.038402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:09.038402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:09.038402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:09.038402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:09.038402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:09.039028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.039028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:09.039779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.039779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:09.039779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:09.039779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:09.039779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:09.039779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:09.039779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:09.039779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:09.039779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:09.039779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:09.039779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:09.039779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:09.039779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:09.039779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:09.039779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:09.039779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:09.040223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:46:09.040223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:09.040223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:09.040223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:09.040223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:09.040223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:09.040223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:09.040223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:09.040223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:09.040223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:09.040223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:09.040223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:09.040223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:09.040223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:09.040223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:09.040223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:09.040223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:09.040223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:09.040223 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:09.040742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:09.040742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.040742 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:09.046580 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:09.047127 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:09.047362 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:09.047665 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:46:09.048429 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:09.048887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.049892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:09.049892 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:09.049892 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:09.049892 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:09.049892 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:09.049892 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:09.049892 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.049892 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:09.051174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.051174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:46:09.051174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:09.051174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:09.051174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:09.051174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:09.051174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:09.051174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:09.051174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:09.051174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:09.051174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:09.051174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:46:09.051174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:09.051174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:46:09.051174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:46:09.051939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:09.051939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:46:09.051939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:09.051939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:46:09.051939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:46:09.051939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:09.051939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:46:09.051939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.051939 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:46:09.052589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.052589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:09.052589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:09.052589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:09.052589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:09.052589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:46:09.052589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:46:09.052589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:46:09.052589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:09.052589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:09.052589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:09.052589 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:09.053258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:09.053258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:09.053258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:09.053258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:09.053258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:09.053258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:09.053258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:09.053258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:09.053258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:09.053258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:09.053258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:09.053258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:09.053258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:09.053258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:09.053917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:09.053917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:09.053917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.053917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:09.054470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.054470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:09.054470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:09.054470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:09.054470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:09.054470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:09.054470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:09.054470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:09.054470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:09.054470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:09.054470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:09.054470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:09.054470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:09.054470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:09.054470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:09.054470 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:09.055274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:46:09.055274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:46:09.055274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:46:09.055274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:09.055274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:09.055274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:09.055274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:09.055274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:09.055274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:09.055274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:09.055274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:09.055274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:09.055274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:09.055274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:09.055274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:09.055274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:09.055274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:09.055274 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:09.056136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:09.056136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:09.056136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:09.056136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:09.056136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:09.056136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.056136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:09.067315 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:09.067689 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:09.067867 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:09.068037 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:09.068609 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:09.069187 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.070320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:09.070320 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:09.070320 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:09.070320 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:09.070320 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:09.070320 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:09.070320 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.070320 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:09.072136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.072136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:46:09.072136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:09.072136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:09.072136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:09.072136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:09.072136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:09.072136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:09.072136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:09.072136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:09.072136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:09.072136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:46:09.072136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:09.072136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:46:09.072136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:46:09.072947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:09.072947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:46:09.072947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:09.072947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:46:09.072947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:46:09.072947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:09.072947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:46:09.072947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.072947 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:46:09.074252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.074252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:09.074252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:09.074252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:09.074252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:09.074252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:46:09.074252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:46:09.074252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:09.074252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:09.074252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:09.074252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:09.074862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:09.074862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:09.074862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:09.074862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:09.074862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:09.074862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:09.074862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:09.074862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:09.074862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:09.074862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:09.074862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:09.074862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:09.074862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:09.074862 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:09.075537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:09.075537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:09.075537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.075537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:09.076195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.076195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:09.076195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:09.076195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:09.076195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:09.076195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:09.076195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:09.076195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:09.076195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:09.076195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:09.076195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:09.076195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:09.076195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:09.076195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:09.076195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:09.076195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:09.076195 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:46:09.076978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:46:09.076978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:09.076978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:09.076978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:09.076978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:09.076978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:09.076978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:09.076978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:09.076978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:09.076978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:09.076978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:09.076978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:09.076978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:09.076978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:09.076978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:09.076978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:09.076978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:09.076978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:09.076978 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:09.077438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:09.077438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:09.077438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.077438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:09.145517 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:09.146059 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:09.146187 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:09.146433 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:09.147065 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:09.147525 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:09.147881 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.148809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:09.148809 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:09.148809 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:09.148809 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:09.148809 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:09.148809 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:09.148809 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.148809 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:09.149809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.149809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:09.149809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:09.149809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:09.149809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:09.149809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:46:09.149809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:09.149809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:09.149809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:09.149809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:09.149809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:09.149809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:09.150174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:09.150174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:09.150174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:09.150174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:09.150174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:09.150174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:09.150174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:09.150174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:09.150174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:09.150174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:09.150174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:09.150174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:09.150174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:09.150518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.150518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:09.150903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.150903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:09.150903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:09.150903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:09.150903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:09.150903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:09.150903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:09.150903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:09.150903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:09.150903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:09.150903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:09.150903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:09.150903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:09.150903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:09.150903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:09.150903 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:09.151336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:46:09.151336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:09.151336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:09.151336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:09.151336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:09.151336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:09.151336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:09.151336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:09.151336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:09.151336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:09.151336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:09.151336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:09.151336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:09.151336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:09.151336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:09.151336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:09.151336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:09.151336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:09.151336 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:09.151856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:09.151856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:09.151856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:13.057429 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:13.058039 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_for_inconsistencies {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:13.058238 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Checking Neutron and OVN revision consistency. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:13.069805 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:440}}
Apr 28 20:46:13.069976 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Number of inconsistencies found at create/update: networks=4, routers=1, router_ports=1 {{(pid=106506) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:419}}
Apr 28 20:46:13.070074 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 7ff08466-af6d-4e13-bb1d-8987170a5734 (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:46:13.109902 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-7ff08466-af6d-4e13-bb1d-8987170a5734, resource={'id': '7ff08466-af6d-4e13-bb1d-8987170a5734', 'name': 'private', 'tenant_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'project_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['4b461bd9-09ea-4f87-a007-a11ab022ff7f', '8f3677a2-e2be-443a-a90c-2cd7b3095c49'], 'standard_attr_id': 12, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-28T20:10:58Z', 'updated_at': '2026-04-28T20:11:02Z', 'revision_number': 3, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 46342}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:13.110132 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-7ff08466-af6d-4e13-bb1d-8987170a5734, col_values=(('external_ids', {'neutron:network_name': 'private', 'neutron:mtu': '1372', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:13.110386 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-27fbc209-4b05-4b4a-8d5c-40d6d87b7e22, external_ids_update=None, columns={'tag': 46342}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:13.110511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.111156 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:13.111156 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:13.111156 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:13.111156 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:13.111156 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:13.111156 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:13.111156 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.111156 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:13.112465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 7ff08466-af6d-4e13-bb1d-8987170a5734 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.112465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:13.112465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:46:13.112465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:46:13.112465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:46:13.112465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:46:13.112465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:46:13.112465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:13.112465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:13.112465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:13.112465 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:13.112744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:13.112744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:13.112744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:13.112744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:13.112744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:13.112744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:13.112744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:13.112744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:13.112744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:13.112744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:13.112744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:13.112744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:13.112744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:13.112744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:13.113036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:13.113036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.113036 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:13.113036 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 8a1fb774-5693-4139-8b75-6ab5273228c7 (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:46:13.145225 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, resource={'id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'name': 'public', 'tenant_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'project_id': '8913d11b880a4aa4a6c48a207b58c9c5', 'admin_state_up': True, 'mtu': 1430, 'status': 'ACTIVE', 'subnets': ['335d12ce-2d0c-4086-b3f4-2af390c03d70', '5308c10f-28d1-441c-acb2-d1d6dc3d2ece'], 'standard_attr_id': 18, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': True, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'is_default': True, 'tags': [], 'created_at': '2026-04-28T20:11:05Z', 'updated_at': '2026-04-28T20:11:13Z', 'revision_number': 3, 'provider:network_type': 'flat', 'provider:physical_network': 'public', 'provider:segmentation_id': None}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:13.145547 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, col_values=(('external_ids', {'neutron:network_name': 'public', 'neutron:mtu': '1430', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'flat', 'neutron:provnet-physical-network': 'public'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false', 'fdb_age_threshold': '300', 'broadcast-arps-to-all-routers': 'true'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:13.146184 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-7dbf713e-d963-4884-ae64-da4ce196e719, external_ids_update=None, columns={'tag': []}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:13.146526 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.147534 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:13.147534 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:13.147534 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:13.147534 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:13.147534 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:13.147534 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:13.147534 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.147534 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:13.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 8a1fb774-5693-4139-8b75-6ab5273228c7 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:13.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:46:13.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:46:13.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:46:13.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:46:13.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:46:13.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:13.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:13.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:13.148948 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:13.149206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:13.149206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:13.149206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:13.149206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:13.149206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:13.149206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:13.149206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:13.149206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:13.149206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:13.149206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:13.149206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:13.149206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:13.149206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:13.149206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:13.149455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:13.149455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.149455 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:13.149455 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource c6718637-daa9-4db1-bd11-63abe3891db6 (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:46:13.181513 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-c6718637-daa9-4db1-bd11-63abe3891db6, resource={'id': 'c6718637-daa9-4db1-bd11-63abe3891db6', 'name': 'admin_net', 'tenant_id': '3eb96b75a3044dd0a338616e1d9081d0', 'project_id': '3eb96b75a3044dd0a338616e1d9081d0', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['c1cc9883-a7e2-4abc-8892-a7c1b200d3ad'], 'standard_attr_id': 42, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-28T20:15:13Z', 'updated_at': '2026-04-28T20:15:14Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 7626}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:13.181818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-c6718637-daa9-4db1-bd11-63abe3891db6, col_values=(('external_ids', {'neutron:network_name': 'admin_net', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:13.182160 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-7461d8df-a8a0-40c3-87a1-26ed88f9f7ce, external_ids_update=None, columns={'tag': 7626}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:13.182286 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.183058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:13.183058 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:13.183058 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:13.183058 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:13.183058 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:13.183058 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:13.183058 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.183058 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:13.184446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource c6718637-daa9-4db1-bd11-63abe3891db6 (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.184446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:13.184446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:46:13.184446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:46:13.184446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:46:13.184446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:46:13.184446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:46:13.184446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:13.184446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:13.184446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:13.184446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:13.184909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:13.184909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:13.184909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:13.184909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:13.184909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:13.184909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:13.184909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:13.184909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:13.184909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:13.184909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:13.184909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:13.184909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:13.184909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:13.184909 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:13.185172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:13.185172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.185172 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:13.185172 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource d3dfebbd-7943-4813-b509-739084ada63b (type: networks) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:46:13.215121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d3dfebbd-7943-4813-b509-739084ada63b, resource={'id': 'd3dfebbd-7943-4813-b509-739084ada63b', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-pi48i', 'tenant_id': 'a11fcd8844504e7eaf509ece683e8c8f', 'project_id': 'a11fcd8844504e7eaf509ece683e8c8f', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['c68bc19b-c88a-4cd2-bf3a-d847d267d23f'], 'standard_attr_id': 52, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2026-04-28T20:15:49Z', 'updated_at': '2026-04-28T20:15:50Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 58143}, resource_type=networks, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:13.215396 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-d3dfebbd-7943-4813-b509-739084ada63b, col_values=(('external_ids', {'neutron:network_name': 'k8s-clusterapi-cluster-magnum-system-kube-pi48i', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve'}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:13.215572 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-ed33b0a1-b0b7-4e33-aa96-92c114a82f5d, external_ids_update=None, columns={'tag': 58143}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:13.215654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.216283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:13.216283 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:13.216283 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:13.216283 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:13.216283 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:13.216283 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:13.216283 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.216283 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:13.217189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource d3dfebbd-7943-4813-b509-739084ada63b (type: networks): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.217189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:13.217189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:46:13.217189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:46:13.217189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:46:13.217189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:46:13.217189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 2327, in update_network
Apr 28 20:46:13.217189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:13.217189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:13.217189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:13.217189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:13.217428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:13.217428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:13.217428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:13.217428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:13.217428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:13.217428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:13.217428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:13.217428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:13.217428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:13.217428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:13.217428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:13.217428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:13.217428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:13.217428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:13.217699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:13.217699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.217699 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:13.217699 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 6feb526a-13c6-41ac-983d-ea1d2251ff3d (type: routers) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:46:13.311820 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, resource={'id': '6feb526a-13c6-41ac-983d-ea1d2251ff3d', 'name': 'k8s-clusterapi-cluster-magnum-system-kube-pi48i', 'project_id': 'a11fcd8844504e7eaf509ece683e8c8f', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'external_fixed_ips': [{'subnet_id': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece', 'ip_address': '172.24.5.87'}, {'subnet_id': '335d12ce-2d0c-4086-b3f4-2af390c03d70', 'ip_address': '2001:db8::2d8'}], 'enable_snat': True}, 'gw_port_id': '73493630-b0ac-48ec-be98-5b048d2e2ce7', 'description': 'Created by cluster-api-provider-openstack cluster magnum-system-kube-pi48i', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'external_fixed_ips': [{'ip_address': '172.24.5.87', 'subnet_id': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece'}, {'ip_address': '2001:db8::2d8', 'subnet_id': '335d12ce-2d0c-4086-b3f4-2af390c03d70'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2026-04-28T20:15:51Z', 'updated_at': '2026-04-28T20:15:54Z', 'revision_number': 3, 'tenant_id': 'a11fcd8844504e7eaf509ece683e8c8f'}, resource_type=routers, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:13.312188 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, columns={'external_ids': {'neutron:router_name': 'k8s-clusterapi-cluster-magnum-system-kube-pi48i', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:13.312188 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "73493630-b0ac-48ec-be98-5b048d2e2ce7"')], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:13.312188 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-8a1fb774-5693-4139-8b75-6ab5273228c7, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "73493630-b0ac-48ec-be98-5b048d2e2ce7"')], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:13.312188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.312188 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:13.312188 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:13.312188 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:13.312188 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:13.312402 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:13.312402 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:13.312402 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.312402 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:13.312402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Unable to update router 6feb526a-13c6-41ac-983d-ea1d2251ff3d. Error: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.312793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 6feb526a-13c6-41ac-983d-ea1d2251ff3d (type: routers): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.312793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:13.312793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:46:13.312793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:46:13.312793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:46:13.312793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:46:13.312793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1634, in update_router
Apr 28 20:46:13.312793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with excutils.save_and_reraise_exception():
Apr 28 20:46:13.312793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 271, in __exit__
Apr 28 20:46:13.312793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.force_reraise()
Apr 28 20:46:13.312793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/oslo_utils/excutils.py", line 233, in force_reraise
Apr 28 20:46:13.313028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise self.value
Apr 28 20:46:13.313028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1568, in update_router
Apr 28 20:46:13.313028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:13.313028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:13.313028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:13.313028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:13.313028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:13.313028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:13.313028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:13.313028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:13.313028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:13.313028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:13.313028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:13.313028 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:13.313283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:13.313283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:13.313283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:13.313283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:13.313283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:13.313283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:13.313283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:13.313283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.313283 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:13.313283 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Fixing resource 73493630-b0ac-48ec-be98-5b048d2e2ce7 (type: router_ports) at create/update {{(pid=106506) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:449}}
Apr 28 20:46:13.429739 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-73493630-b0ac-48ec-be98-5b048d2e2ce7, resource={'id': '73493630-b0ac-48ec-be98-5b048d2e2ce7', 'name': '', 'network_id': '8a1fb774-5693-4139-8b75-6ab5273228c7', 'project_id': '', 'mac_address': 'fa:16:3e:f8:07:38', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '6feb526a-13c6-41ac-983d-ea1d2251ff3d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 57, 'fixed_ips': [{'subnet_id': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece', 'ip_address': '172.24.5.87'}, {'subnet_id': '335d12ce-2d0c-4086-b3f4-2af390c03d70', 'ip_address': '2001:db8::2d8'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0000171185', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'ovs_create_tap': False, 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2026-04-28T20:15:51Z', 'updated_at': '2026-04-28T20:15:54Z', 'revision_number': 5, 'tenant_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:13.429985 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-73493630-b0ac-48ec-be98-5b048d2e2ce7, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': '5308c10f-28d1-441c-acb2-d1d6dc3d2ece 335d12ce-2d0c-4086-b3f4-2af390c03d70', 'neutron:network_name': 'neutron-8a1fb774-5693-4139-8b75-6ab5273228c7', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.87/24', '2001:db8::2d8/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:13.430545 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=73493630-b0ac-48ec-be98-5b048d2e2ce7, lrouter_port=lrp-73493630-b0ac-48ec-be98-5b048d2e2ce7, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:13.430941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.431883 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:13.431883 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:13.431883 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:13.431883 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:13.431883 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:13.431883 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:13.431883 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.431883 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:13.433023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Maintenance task: Failed to fix resource 73493630-b0ac-48ec-be98-5b048d2e2ce7 (type: router_ports): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.433023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:13.433023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 462, in check_for_inconsistencies
Apr 28 20:46:13.433023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._fix_create_update(admin_context, row)
Apr 28 20:46:13.433023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 370, in _fix_create_update
Apr 28 20:46:13.433023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     res_map['ovn_update'](context, n_obj)
Apr 28 20:46:13.433023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/ovn_client.py", line 1971, in update_router_port
Apr 28 20:46:13.433023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:13.433023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:13.433023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:13.433023 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:13.434489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:13.434489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:13.434489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:13.434489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:13.434489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:13.434489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:13.434489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:13.434489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:13.434489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:13.434489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:13.434489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:13.434489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:13.434489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:13.434489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:13.435740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:13.435740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.435740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:13.435740 np0000171185 neutron-ovn-maintenance-worker[106506]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_for_inconsistencies finished in 0.375 seconds
Apr 28 20:46:13.817219 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:13.817716 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:13.817949 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:13.818653 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:13.850731 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:13.851063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.852014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:13.852014 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:13.852014 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:13.852014 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:13.852014 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:13.852014 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:13.852014 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.852014 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:13.852634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.852634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:13.852634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:13.852634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:13.852634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:13.852634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:46:13.852634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:13.852634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:13.852634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:13.852634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:13.852634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:13.852634 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:13.853386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:13.853386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:13.853386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:13.853386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:13.853386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:13.853386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:13.853386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:13.853386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:13.853386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:13.853386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:13.853386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:13.853386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:13.853386 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:13.854442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.854442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:13.854442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.854442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:13.854442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:13.854442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:13.854442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:13.854442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:13.854442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:13.854442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:13.854442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:13.854442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:13.854442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:13.854442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:13.854442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:13.855622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:13.855622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:13.855622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:13.855622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:46:13.855622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:13.855622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:13.855622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:13.855622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:13.855622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:13.855622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:13.855622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:13.855622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:13.855622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:13.855622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:13.855622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:13.855622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:13.855622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:13.855622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:13.855622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:13.856580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:13.856580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:13.856580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:13.856580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:13.856580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:13.856580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:14.033303 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:14.033760 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:14.033875 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:14.034857 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:14.035185 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:14.035428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.036173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:14.036173 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:14.036173 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:14.036173 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:14.036173 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:14.036173 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:14.036173 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.036173 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:14.037664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.037664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:14.037664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:14.037664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:14.037664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:14.037664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:46:14.037664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:14.037664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:14.037664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:14.037664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:14.037664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:14.037664 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:14.038063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:14.038063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:14.038063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:14.038063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:14.038063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:14.038063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:14.038063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:14.038063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:14.038063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:14.038063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:14.038063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:14.038063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:14.038063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:14.038387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.038387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:14.039411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.039411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:14.039411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:14.039411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:14.039411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:14.039411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:14.039411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:14.039411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:14.039411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:14.039411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:14.039411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:14.039411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:14.039411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:14.039411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:14.039411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:14.039411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:14.039915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:46:14.039915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:14.039915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:14.039915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:14.039915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:14.039915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:14.039915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:14.039915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:14.039915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:14.039915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:14.039915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:14.039915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:14.039915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:14.039915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:14.039915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:14.039915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:14.039915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:14.039915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:14.039915 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:14.040443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:14.040443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.040443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:14.047113 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:14.047341 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:14.047482 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:14.047566 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:46:14.047913 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:14.048162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.048757 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:14.048757 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:14.048757 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:14.048757 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:14.048757 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:14.048757 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:14.048757 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.048757 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:14.049877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.049877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:46:14.049877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:14.049877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:14.049877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:14.049877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:14.049877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:14.049877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:14.049877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:14.049877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:14.049877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:14.049877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:46:14.049877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:14.049877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:46:14.049877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:46:14.050472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:14.050472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:46:14.050472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:14.050472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:46:14.050472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:46:14.050472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:14.050472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:46:14.050472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.050472 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:46:14.051141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.051141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:14.051141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:14.051141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:14.051141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:14.051141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:46:14.051141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:46:14.051141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:46:14.051141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:14.051141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:14.051141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:14.051141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:14.051422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:14.051422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:14.051422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:14.051422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:14.051422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:14.051422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:14.051422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:14.051422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:14.051422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:14.051422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:14.051422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:14.051422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:14.051422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:14.051422 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:14.051666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:14.051666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:14.051666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.051666 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:14.052735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.052735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:14.052735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:14.052735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:14.052735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:14.052735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:14.052735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:14.052735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:14.052735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:14.052735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:14.052735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:14.052735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:14.052735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:14.052735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:14.052735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:14.052735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:14.053844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:46:14.053844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:46:14.053844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:46:14.053844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:14.053844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:14.053844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:14.053844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:14.053844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:14.053844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:14.053844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:14.053844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:14.053844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:14.053844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:14.053844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:14.053844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:14.053844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:14.053844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:14.053844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:14.054726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:14.054726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:14.054726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:14.054726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:14.054726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:14.054726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.054726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:14.067750 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:14.068453 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:14.068453 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:14.068582 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:14.069068 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:14.069507 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.070559 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:14.070559 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:14.070559 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:14.070559 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:14.070559 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:14.070559 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:14.070559 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.070559 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:14.071779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.071779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:46:14.071779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:14.071779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:14.071779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:14.071779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:14.071779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:14.071779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:14.071779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:14.071779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:14.071779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:14.071779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:46:14.071779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:14.071779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:46:14.071779 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:46:14.072537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:14.072537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:46:14.072537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:14.072537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:46:14.072537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:46:14.072537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:14.072537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:46:14.072537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.072537 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:46:14.073404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.073404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:14.073404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:14.073404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:14.073404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:14.073404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:46:14.073404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:46:14.073404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:14.073404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:14.073404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:14.073404 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:14.074758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:14.074758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:14.074758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:14.074758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:14.074758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:14.074758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:14.074758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:14.074758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:14.074758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:14.074758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:14.074758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:14.074758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:14.074758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:14.074758 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:14.076297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:14.076297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:14.076297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.076297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:14.076297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.076297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:14.076297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:14.076297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:14.076297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:14.076297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:14.076297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:14.076297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:14.076297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:14.076297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:14.077402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:14.077402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:14.077402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:14.077402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:14.077402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:14.077402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:14.077402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:46:14.077402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:46:14.077402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:14.077402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:14.077402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:14.077402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:14.077402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:14.077402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:14.077402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:14.077402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:14.077402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:14.077402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:14.077402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:14.078968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:14.078968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:14.078968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:14.078968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:14.078968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:14.078968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:14.078968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:14.078968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:14.078968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:14.078968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.078968 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:14.146074 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:14.146511 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:14.146712 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:14.147012 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:14.147497 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:14.147879 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:14.148093 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.148879 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:14.148879 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:14.148879 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:14.148879 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:14.148879 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:14.148879 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:14.148879 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.148879 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:14.150291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.150291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:14.150291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:14.150291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:14.150291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:14.150291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:46:14.150291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:14.150291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:14.150291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:14.150291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:14.150291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:14.150291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:14.150808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:14.150808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:14.150808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:14.150808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:14.150808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:14.150808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:14.150808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:14.150808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:14.150808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:14.150808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:14.150808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:14.150808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:14.150808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:14.151183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.151183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:14.151863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.151863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:14.151863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:14.151863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:14.151863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:14.151863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:14.151863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:14.151863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:14.151863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:14.151863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:14.151863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:14.151863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:14.151863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:14.151863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:14.151863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:14.151863 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:14.152355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:46:14.152355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:14.152355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:14.152355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:14.152355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:14.152355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:14.152355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:14.152355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:14.152355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:14.152355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:14.152355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:14.152355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:14.152355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:14.152355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:14.152355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:14.152355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:14.152355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:14.152355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:14.152355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:14.152928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:14.152928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:14.152928 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:18.817947 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:18.818603 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:18.818833 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:18.820130 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:18.882985 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:18.883521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:18.884843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:18.884843 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:18.884843 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:18.884843 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:18.884843 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:18.884843 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:18.884843 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:18.884843 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:18.887090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.066 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:18.887090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:18.887090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:18.887090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:18.887090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:18.887090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:46:18.887090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:18.887090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:18.887090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:18.887090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:18.887090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:18.887090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:18.888099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:18.888099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:18.888099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:18.888099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:18.888099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:18.888099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:18.888099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:18.888099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:18.888099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:18.888099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:18.888099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:18.888099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:18.888099 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:18.889128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:18.889128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:18.889128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:18.889128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:18.889128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:18.889128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:18.889128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:18.889128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:18.889128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:18.889128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:18.889128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:18.889128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:18.889128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:18.889128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:18.889128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:18.890258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:18.890258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:18.890258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:18.890258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:46:18.890258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:18.890258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:18.890258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:18.890258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:18.890258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:18.890258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:18.890258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:18.890258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:18.890258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:18.890258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:18.890258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:18.890258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:18.890258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:18.890258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:18.890258 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:18.891109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:18.891109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:18.891109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:18.891109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:18.891109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:18.891109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:19.034479 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:19.035006 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:19.035212 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:19.036301 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:19.036635 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:19.036904 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.037826 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:19.037826 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:19.037826 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:19.037826 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:19.037826 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:19.037826 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:19.037826 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.037826 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:19.039293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.039293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:19.039293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:19.039293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:19.039293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:19.039293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:46:19.039293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:19.039293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:19.039293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:19.039293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:19.039293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:19.039293 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:19.039701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:19.039701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:19.039701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:19.039701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:19.039701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:19.039701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:19.039701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:19.039701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:19.039701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:19.039701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:19.039701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:19.039701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:19.039701 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:19.040067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.040067 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:19.040843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.040843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:19.040843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:19.040843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:19.040843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:19.040843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:19.040843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:19.040843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:19.040843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:19.040843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:19.040843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:19.040843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:19.040843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:19.040843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:19.040843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:19.040843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:19.041315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:46:19.041315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:19.041315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:19.041315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:19.041315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:19.041315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:19.041315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:19.041315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:19.041315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:19.041315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:19.041315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:19.041315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:19.041315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:19.041315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:19.041315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:19.041315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:19.041315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:19.041315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:19.041315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:19.041929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:19.041929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.041929 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:19.047607 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:19.047948 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:19.048034 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:19.048159 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:46:19.048663 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:19.048990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.049744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:19.049744 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:19.049744 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:19.049744 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:19.049744 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:19.049744 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:19.049744 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.049744 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:19.050571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.050571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:46:19.050571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:19.050571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:19.050571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:19.050571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:19.050571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:19.050571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:19.050571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:19.050571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:19.050571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:19.050571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:46:19.050571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:19.050571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:46:19.050571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:46:19.051101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:19.051101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:46:19.051101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:19.051101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:46:19.051101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:46:19.051101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:19.051101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:46:19.051101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.051101 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:46:19.051411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.051411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:19.051411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:19.051411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:19.051411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:19.051411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:46:19.051411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:46:19.051411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:46:19.051411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:19.051411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:19.051411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:19.051411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:19.051796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:19.051796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:19.051796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:19.051796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:19.051796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:19.051796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:19.051796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:19.051796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:19.051796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:19.051796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:19.051796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:19.051796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:19.051796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:19.051796 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:19.052143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:19.052143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:19.052143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.052143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:19.052345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.052345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:19.052345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:19.052345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:19.052345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:19.052345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:19.052345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:19.052345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:19.052345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:19.052345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:19.052345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:19.052345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:19.052345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:19.052345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:19.052345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:19.052345 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:19.052795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:46:19.052795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:46:19.052795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:46:19.052795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:19.052795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:19.052795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:19.052795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:19.052795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:19.052795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:19.052795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:19.052795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:19.052795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:19.052795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:19.052795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:19.052795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:19.052795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:19.052795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:19.052795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:19.053313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:19.053313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:19.053313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:19.053313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:19.053313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:19.053313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.053313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:19.068302 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:19.068608 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:19.068788 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:19.068979 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:19.069905 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:19.070276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.071065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:19.071065 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:19.071065 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:19.071065 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:19.071065 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:19.071065 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:19.071065 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.071065 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:19.071891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.071891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:46:19.071891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:19.071891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:19.071891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:19.071891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:19.071891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:19.071891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:19.071891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:19.071891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:19.071891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:19.071891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:46:19.071891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:19.071891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:46:19.071891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:46:19.072573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:19.072573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:46:19.072573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:19.072573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:46:19.072573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:46:19.072573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:19.072573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:46:19.072573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.072573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:46:19.072969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.072969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:19.072969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:19.072969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:19.072969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:19.072969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:46:19.072969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:46:19.072969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:19.072969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:19.072969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:19.072969 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:19.073473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:19.073473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:19.073473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:19.073473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:19.073473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:19.073473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:19.073473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:19.073473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:19.073473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:19.073473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:19.073473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:19.073473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:19.073473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:19.073473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:19.074142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:19.074142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:19.074142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.074142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:19.074142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.074142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:19.074142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:19.074142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:19.074142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:19.074142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:19.074142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:19.074142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:19.074142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:19.074142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:19.075318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:19.075318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:19.075318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:19.075318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:19.075318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:19.075318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:19.075318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:46:19.075318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:46:19.075318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:19.075318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:19.075318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:19.075318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:19.075318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:19.075318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:19.075318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:19.075318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:19.075318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:19.075318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:19.075318 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:19.076142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:19.076142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:19.076142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:19.076142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:19.076142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:19.076142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:19.076142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:19.076142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:19.076142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:19.076142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.076142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:19.146651 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:19.147168 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:19.147254 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:19.147399 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:19.148025 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:19.148454 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:19.148713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.149351 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:19.149351 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:19.149351 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:19.149351 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:19.149351 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:19.149351 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:19.149351 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.149351 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:19.150071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.150071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:19.150071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:19.150071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:19.150071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:19.150071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:46:19.150071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:19.150071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:19.150071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:19.150071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:19.150071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:19.150071 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:19.150795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:19.150795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:19.150795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:19.150795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:19.150795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:19.150795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:19.150795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:19.150795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:19.150795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:19.150795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:19.150795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:19.150795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:19.150795 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:19.151378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.151378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:19.151378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.151378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:19.151378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:19.151378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:19.151378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:19.151378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:19.151378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:19.151378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:19.151378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:19.151378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:19.151378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:19.151378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:19.151378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:19.152100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:19.152100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:19.152100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:19.152100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:46:19.152100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:19.152100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:19.152100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:19.152100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:19.152100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:19.152100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:19.152100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:19.152100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:19.152100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:19.152100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:19.152100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:19.152100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:19.152100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:19.152100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:19.152100 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:19.153143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:19.153143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:19.153143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:19.153143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:19.153143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:19.153143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:23.818726 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:23.819966 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:23.819966 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:23.821283 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:23.863281 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:23.863697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:23.864695 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:23.864695 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:23.864695 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:23.864695 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:23.864695 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:23.864695 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:23.864695 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:23.864695 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:23.865803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:23.865803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:23.865803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:23.865803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:23.865803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:23.865803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:46:23.865803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:23.865803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:23.865803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:23.865803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:23.865803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:23.865803 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:23.866451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:23.866451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:23.866451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:23.866451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:23.866451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:23.866451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:23.866451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:23.866451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:23.866451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:23.866451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:23.866451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:23.866451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:23.866451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:23.866945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:23.866945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:23.866945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:23.866945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:23.866945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:23.866945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:23.866945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:23.866945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:23.866945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:23.866945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:23.866945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:23.866945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:23.866945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:23.866945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:23.866945 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:23.867402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:23.867402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:23.867402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:23.867402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:46:23.867402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:23.867402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:23.867402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:23.867402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:23.867402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:23.867402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:23.867402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:23.867402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:23.867402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:23.867402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:23.867402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:23.867402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:23.867402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:23.867402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:23.867402 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:23.867983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:23.867983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:23.867983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:23.867983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:23.867983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:23.867983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:24.035113 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:24.035986 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:24.036169 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:24.037644 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:24.038165 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:24.038519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.039808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:24.039808 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:24.039808 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:24.039808 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:24.039808 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:24.039808 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:24.039808 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.039808 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:24.041019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.041019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:24.041019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:24.041019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:24.041019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:24.041019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:46:24.041019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:24.041019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:24.041019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:24.041019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:24.041019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:24.041019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:24.041276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:24.041276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:24.041276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:24.041276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:24.041276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:24.041276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:24.041276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:24.041276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:24.041276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:24.041276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:24.041276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:24.041276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:24.041276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:24.041573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.041573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:24.042144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.042144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:24.042144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:24.042144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:24.042144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:24.042144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:24.042144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:24.042144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:24.042144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:24.042144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:24.042144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:24.042144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:24.042144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:24.042144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:24.042144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:24.042144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:24.042861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:46:24.042861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:24.042861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:24.042861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:24.042861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:24.042861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:24.042861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:24.042861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:24.042861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:24.042861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:24.042861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:24.042861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:24.042861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:24.042861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:24.042861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:24.042861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:24.042861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:24.042861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:24.042861 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:24.043562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:24.043562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.043562 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:24.048155 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:24.048611 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:24.048611 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:24.048747 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:46:24.049315 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:24.049556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.050197 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:24.050197 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:24.050197 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:24.050197 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:24.050197 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:24.050197 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:24.050197 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.050197 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:24.050920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.050920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:46:24.050920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:24.050920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:24.050920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:24.050920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:24.050920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:24.050920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:24.050920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:24.050920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:24.050920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:24.050920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:46:24.050920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:24.050920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:46:24.050920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:46:24.051250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:24.051250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:46:24.051250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:24.051250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:46:24.051250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:46:24.051250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:24.051250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:46:24.051250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.051250 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:46:24.051533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.051533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:24.051533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:24.051533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:24.051533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:24.051533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:46:24.051533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:46:24.051533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:46:24.051533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:24.051533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:24.051533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:24.051533 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:24.052783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:24.052783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:24.052783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:24.052783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:24.052783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:24.052783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:24.052783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:24.052783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:24.052783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:24.052783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:24.052783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:24.052783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:24.052783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:24.052783 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:24.053034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:24.053034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:24.053034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.053034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:24.053034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.053034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:24.053034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:24.053034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:24.053034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:24.053034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:24.053034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:24.053034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:24.053034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:24.053034 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:24.053285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:24.053285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:24.053285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:24.053285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:24.053285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:24.053285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:24.053285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:46:24.053285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:46:24.053285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:46:24.053285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:24.053285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:24.053285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:24.053285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:24.053285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:24.053285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:24.053285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:24.053285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:24.053285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:24.053285 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:24.053628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:24.053628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:24.053628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:24.053628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:24.053628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:24.053628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:24.053628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:24.053628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:24.053628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:24.053628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:24.053628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.053628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:24.069013 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:24.069477 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:24.069808 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:24.069808 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:24.070518 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:24.070918 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.071655 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:24.071655 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:24.071655 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:24.071655 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:24.071655 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:24.071655 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:24.071655 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.071655 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:24.072762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.072762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:46:24.072762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:24.072762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:24.072762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:24.072762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:24.072762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:24.072762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:24.072762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:24.072762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:24.072762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:24.072762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:46:24.072762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:24.072762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:46:24.072762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:46:24.073206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:24.073206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:46:24.073206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:24.073206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:46:24.073206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:46:24.073206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:24.073206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:46:24.073206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.073206 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:46:24.073750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.073750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:24.073750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:24.073750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:24.073750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:24.073750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:46:24.073750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:46:24.073750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:24.073750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:24.073750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:24.073750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:24.074110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:24.074110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:24.074110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:24.074110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:24.074110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:24.074110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:24.074110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:24.074110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:24.074110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:24.074110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:24.074110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:24.074110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:24.074110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:24.074110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:24.074513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:24.074513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:24.074513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.074513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:24.075041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.075041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:24.075041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:24.075041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:24.075041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:24.075041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:24.075041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:24.075041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:24.075041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:24.075041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:24.075041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:24.075041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:24.075041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:24.075041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:24.075041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:24.075041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:24.075041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:46:24.075601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:46:24.075601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:24.075601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:24.075601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:24.075601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:24.075601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:24.075601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:24.075601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:24.075601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:24.075601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:24.075601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:24.075601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:24.075601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:24.075601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:24.075601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:24.075601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:24.075601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:24.075601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:24.075601 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:24.076221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:24.076221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:24.076221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.076221 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:24.147345 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:24.148994 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:24.149149 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:24.149526 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:24.150931 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:24.151405 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:24.151660 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.153247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:24.153247 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:24.153247 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:24.153247 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:24.153247 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:24.153247 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:24.153247 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.153247 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:24.154474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.154474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:24.154474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:24.154474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:24.154474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:24.154474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:46:24.154474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:24.154474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:24.154474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:24.154474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:24.154474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:24.154474 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:24.154927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:24.154927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:24.154927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:24.154927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:24.154927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:24.154927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:24.154927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:24.154927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:24.154927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:24.154927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:24.154927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:24.154927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:24.154927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:24.155329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.155329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:24.155839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.155839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:24.155839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:24.155839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:24.155839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:24.155839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:24.155839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:24.155839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:24.155839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:24.155839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:24.155839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:24.155839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:24.155839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:24.155839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:24.155839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:24.155839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:24.156610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:46:24.156610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:24.156610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:24.156610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:24.156610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:24.156610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:24.156610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:24.156610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:24.156610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:24.156610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:24.156610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:24.156610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:24.156610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:24.156610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:24.156610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:24.156610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:24.156610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:24.156610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:24.156610 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:24.157417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:24.157417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:24.157417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:28.819385 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:28.820021 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:28.820218 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:28.821465 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:28.867582 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:28.867866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:28.868638 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:28.868638 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:28.868638 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:28.868638 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:28.868638 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:28.868638 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:28.868638 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:28.868638 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:28.869761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:28.869761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:28.869761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:28.869761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:28.869761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:28.869761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:46:28.869761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:28.869761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:28.869761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:28.869761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:28.869761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:28.869761 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:28.871090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:28.871090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:28.871090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:28.871090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:28.871090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:28.871090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:28.871090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:28.871090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:28.871090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:28.871090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:28.871090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:28.871090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:28.871090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:28.871090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:28.871090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:28.873150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:28.873150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:28.873150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:28.873150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:46:28.873150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:28.873150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:28.873150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:28.873150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:28.873150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:28.873150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:28.873150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:28.873150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:28.873150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:28.873150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:28.873150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:28.873150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:28.873150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:28.873150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:28.873150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:28.873820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:28.873820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:28.873820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:28.873820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:28.873820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:28.873820 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:29.036238 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:29.036718 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:29.036818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:29.038135 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:29.038417 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:29.038577 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.039591 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:29.039591 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:29.039591 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:29.039591 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:29.039591 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:29.039591 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:29.039591 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.039591 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:29.040154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.040154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:29.040154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:29.040154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:29.040154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:29.040154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:46:29.040154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:29.040154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:29.040154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:29.040154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:29.040154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:29.040154 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:29.040592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:29.040592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:29.040592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:29.040592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:29.040592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:29.040592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:29.040592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:29.040592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:29.040592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:29.040592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:29.040592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:29.040592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:29.040592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:29.041191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.041191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:29.041191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.041191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:29.041191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:29.041191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:29.041191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:29.041191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:29.041191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:29.041191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:29.041191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:29.041191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:29.041191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:29.041191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:29.041191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:29.041705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:29.041705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:29.041705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:29.041705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:46:29.041705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:29.041705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:29.041705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:29.041705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:29.041705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:29.041705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:29.041705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:29.041705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:29.041705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:29.041705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:29.041705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:29.041705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:29.041705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:29.041705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:29.041705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:29.042297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:29.042297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:29.042297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:29.042297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:29.042297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.042297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:29.048706 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:29.049034 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:29.049125 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:29.049184 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:46:29.049853 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:29.050126 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.050791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:29.050791 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:29.050791 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:29.050791 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:29.050791 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:29.050791 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:29.050791 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.050791 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:29.051360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.051360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:46:29.051360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:29.051360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:29.051360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:29.051360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:29.051360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:29.051360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:29.051360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:29.051360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:29.051360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:29.051360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:46:29.051360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:29.051360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:46:29.051360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:46:29.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:29.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:46:29.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:29.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:46:29.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:46:29.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:29.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:46:29.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:46:29.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:29.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:29.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:29.051930 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:29.052443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:46:29.052443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:46:29.052443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:46:29.052443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:29.052443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:29.052443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:29.052443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:29.052443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:29.052443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:29.052443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:29.052443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:29.052443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:29.052443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:29.052893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:29.052893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:29.052893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:29.052893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:29.052893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:29.052893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:29.052893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:29.052893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:29.052893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:29.052893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:29.052893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.052893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:29.053399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.053399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:29.053399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:29.053399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:29.053399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:29.053399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:29.053399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:29.053399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:29.053399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:29.053399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:29.053399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:29.053399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:29.053399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:29.053399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:29.053399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:29.053399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:29.053975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:46:29.053975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:46:29.053975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:46:29.053975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:29.053975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:29.053975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:29.053975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:29.053975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:29.053975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:29.053975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:29.053975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:29.053975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:29.053975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:29.053975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:29.053975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:29.053975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:29.053975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:29.053975 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:29.054868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:29.054868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:29.054868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:29.054868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:29.054868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:29.054868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.054868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:29.069699 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:29.069914 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:29.069914 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:29.070046 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:29.070948 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:29.071387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.072368 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:29.072368 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:29.072368 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:29.072368 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:29.072368 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:29.072368 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:29.072368 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.072368 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:29.072966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.072966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:46:29.072966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:29.072966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:29.072966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:29.072966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:29.072966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:29.072966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:29.072966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:29.072966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:29.072966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:29.072966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:46:29.072966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:29.072966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:46:29.072966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:46:29.073633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:29.073633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:46:29.073633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:29.073633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:46:29.073633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:46:29.073633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:29.073633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:46:29.073633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.073633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:46:29.073633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.073633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:29.073633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:29.073633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:29.073633 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:29.074344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:46:29.074344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:46:29.074344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:29.074344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:29.074344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:29.074344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:29.074344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:29.074344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:29.074344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:29.074344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:29.074344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:29.074344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:29.074344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:29.074966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:29.074966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:29.074966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:29.074966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:29.074966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:29.074966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:29.074966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:29.074966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:29.074966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:29.074966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.074966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:29.074966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.075585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:29.075585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:29.075585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:29.075585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:29.075585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:29.075585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:29.075585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:29.075585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:29.075585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:29.075585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:29.075585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:29.075585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:29.075585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:29.075585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:29.075585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:29.075585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:46:29.075585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:46:29.075585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:29.075585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:29.076488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:29.076488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:29.076488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:29.076488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:29.076488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:29.076488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:29.076488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:29.076488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:29.076488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:29.076488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:29.076488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:29.076488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:29.076488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:29.076488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:29.076488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:29.076488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:29.076488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:29.076488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:29.076488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.076488 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:29.149017 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:29.149525 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:29.149731 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:29.150001 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:29.150658 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:29.151150 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:29.151399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.152264 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:29.152264 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:29.152264 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:29.152264 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:29.152264 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:29.152264 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:29.152264 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.152264 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:29.153174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.153174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:29.153174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:29.153174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:29.153174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:29.153174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:46:29.153174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:29.153174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:29.153174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:29.153174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:29.153174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:29.153174 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:29.153802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:29.153802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:29.153802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:29.153802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:29.153802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:29.153802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:29.153802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:29.153802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:29.153802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:29.153802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:29.153802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:29.153802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:29.153802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:29.154408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.154408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:29.154408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.154408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:29.154408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:29.154408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:29.154408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:29.154408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:29.154408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:29.154408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:29.154408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:29.154408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:29.154408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:29.154408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:29.154408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:29.155116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:29.155116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:29.155116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:29.155116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:46:29.155116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:29.155116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:29.155116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:29.155116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:29.155116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:29.155116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:29.155116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:29.155116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:29.155116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:29.155116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:29.155116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:29.155116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:29.155116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:29.155116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:29.155116 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:29.156002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:29.156002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:29.156002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:29.156002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:29.156002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:29.156002 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:33.819981 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:33.820623 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:33.820736 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:33.821456 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:33.852936 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:33.853389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:33.857716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:33.857716 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:33.857716 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:33.857716 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:33.857716 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:33.857716 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:33.857716 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:33.857716 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:33.857959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.036 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:33.857959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:33.857959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:33.857959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:33.857959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:33.857959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:46:33.857959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:33.857959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:33.857959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:33.857959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:33.857959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:33.857959 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:33.858239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:33.858239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:33.858239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:33.858239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:33.858239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:33.858239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:33.858239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:33.858239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:33.858239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:33.858239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:33.858239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:33.858239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:33.858239 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:33.858515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:33.858515 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:33.859314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:33.859314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:33.859314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:33.859314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:33.859314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:33.859314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:33.859314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:33.859314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:33.859314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:33.859314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:33.859314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:33.859314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:33.859314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:33.859314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:33.859314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:33.859314 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:33.859672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:46:33.859672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:33.859672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:33.859672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:33.859672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:33.859672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:33.859672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:33.859672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:33.859672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:33.859672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:33.859672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:33.859672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:33.859672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:33.859672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:33.859672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:33.859672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:33.859672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:33.859672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:33.859672 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:33.860066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:33.860066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:33.860066 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:34.037140 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:34.037914 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:34.038141 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:34.039485 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:34.039823 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:34.040009 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.040916 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:34.040916 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:34.040916 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:34.040916 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:34.040916 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:34.040916 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:34.040916 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.040916 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:34.042413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.042413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:34.042413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:34.042413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:34.042413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:34.042413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:46:34.042413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:34.042413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:34.042413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:34.042413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:34.042413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:34.042413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:34.042688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:34.042688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:34.042688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:34.042688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:34.042688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:34.042688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:34.042688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:34.042688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:34.042688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:34.042688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:34.042688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:34.042688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:34.042688 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:34.042936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.042936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:34.043565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.043565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:34.043565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:34.043565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:34.043565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:34.043565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:34.043565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:34.043565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:34.043565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:34.043565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:34.043565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:34.043565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:34.043565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:34.043565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:34.043565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:34.043565 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:34.043943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:46:34.043943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:34.043943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:34.043943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:34.043943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:34.043943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:34.043943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:34.043943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:34.043943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:34.043943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:34.043943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:34.043943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:34.043943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:34.043943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:34.043943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:34.043943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:34.043943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:34.043943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:34.043943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:34.044399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:34.044399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.044399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:34.049363 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:34.049916 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:34.050072 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:34.050214 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:46:34.051154 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:34.051518 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.052898 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:34.052898 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:34.052898 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:34.052898 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:34.052898 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:34.052898 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:34.052898 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.052898 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:34.053993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.053993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:46:34.053993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:34.053993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:34.053993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:34.053993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:34.053993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:34.053993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:34.053993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:34.053993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:34.053993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:34.053993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:46:34.053993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:34.053993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:46:34.053993 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:46:34.054446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:34.054446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:46:34.054446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:34.054446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:46:34.054446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:46:34.054446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:34.054446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:46:34.054446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.054446 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:46:34.057374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.057374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:34.057374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:34.057374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:34.057374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:34.057374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:46:34.057374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:46:34.057374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:46:34.057374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:34.057374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:34.057374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:34.057374 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:34.057817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:34.057817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:34.057817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:34.057817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:34.057817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:34.057817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:34.057817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:34.057817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:34.057817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:34.057817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:34.057817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:34.057817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:34.057817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:34.057817 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:34.058333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:34.058333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:34.058333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.058333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:34.058333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.058333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:34.058333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:34.058333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:34.058333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:34.058333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:34.058333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:34.058333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:34.058333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:34.058333 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:34.058690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:34.058690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:34.058690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:34.058690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:34.058690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:34.058690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:34.058690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:46:34.058690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:46:34.058690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:46:34.058690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:34.058690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:34.058690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:34.058690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:34.058690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:34.058690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:34.058690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:34.058690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:34.058690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:34.058690 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:34.059179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:34.059179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:34.059179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:34.059179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:34.059179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:34.059179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:34.059179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:34.059179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:34.059179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:34.059179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:34.059179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.059179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:34.070161 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:34.070751 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:34.070955 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:34.071187 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:34.072215 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:34.072624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.073615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:34.073615 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:34.073615 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:34.073615 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:34.073615 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:34.073615 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:34.073615 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.073615 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:34.075412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.075412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:46:34.075412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:34.075412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:34.075412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:34.075412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:34.075412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:34.075412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:34.075412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:34.075412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:34.075412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:34.075412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:46:34.075412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:34.075412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:46:34.075412 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:46:34.075785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:34.075785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:46:34.075785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:34.075785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:46:34.075785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:46:34.075785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:34.075785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:46:34.075785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.075785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:46:34.077841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.077841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:34.077841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:34.077841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:34.077841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:34.077841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:46:34.077841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:46:34.077841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:34.077841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:34.077841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:34.077841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:34.078143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:34.078143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:34.078143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:34.078143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:34.078143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:34.078143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:34.078143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:34.078143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:34.078143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:34.078143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:34.078143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:34.078143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:34.078143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:34.078143 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:34.078416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:34.078416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:34.078416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.078416 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:34.079631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.079631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:34.079631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:34.079631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:34.079631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:34.079631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:34.079631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:34.079631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:34.079631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:34.079631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:34.079631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:34.079631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:34.079631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:34.079631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:34.079631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:34.079631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:34.079631 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:46:34.080090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:46:34.080090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:34.080090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:34.080090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:34.080090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:34.080090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:34.080090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:34.080090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:34.080090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:34.080090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:34.080090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:34.080090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:34.080090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:34.080090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:34.080090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:34.080090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:34.080090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:34.080090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:34.080090 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:34.080464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:34.080464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:34.080464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.080464 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:34.149615 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:34.150289 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:34.150510 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:34.150846 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:34.151566 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:34.152052 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:34.152207 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.153122 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:34.153122 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:34.153122 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:34.153122 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:34.153122 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:34.153122 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:34.153122 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.153122 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:34.155830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.155830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:34.155830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:34.155830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:34.155830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:34.155830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:46:34.155830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:34.155830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:34.155830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:34.155830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:34.155830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:34.155830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:34.156133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:34.156133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:34.156133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:34.156133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:34.156133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:34.156133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:34.156133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:34.156133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:34.156133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:34.156133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:34.156133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:34.156133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:34.156133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:34.156366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.156366 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:34.158145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.158145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:34.158145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:34.158145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:34.158145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:34.158145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:34.158145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:34.158145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:34.158145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:34.158145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:34.158145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:34.158145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:34.158145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:34.158145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:34.158145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:34.158145 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:34.158484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:46:34.158484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:34.158484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:34.158484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:34.158484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:34.158484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:34.158484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:34.158484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:34.158484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:34.158484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:34.158484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:34.158484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:34.158484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:34.158484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:34.158484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:34.158484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:34.158484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:34.158484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:34.158484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:34.158874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:34.158874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:34.158874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:38.821490 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:38.822139 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:38.822139 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:38.822779 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:38.860713 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:38.860713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:38.860713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:38.860713 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:38.860713 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:38.860713 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:38.860713 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:38.860713 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:38.860713 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:38.860713 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:38.860713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:38.860713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:38.860713 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:38.861130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:38.861130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:38.861130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:46:38.861130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:38.861130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:38.861130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:38.861130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:38.861130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:38.861130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:38.861130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:38.861130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:38.861130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:38.861130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:38.861130 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:38.861393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:38.861393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:38.861393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:38.861393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:38.861393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:38.861393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:38.861393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:38.861393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:38.861393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:38.861393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:38.861393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:38.861393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:38.861630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:38.861630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:38.861630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:38.861630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:38.861630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:38.861630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:38.861630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:38.861630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:38.861630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:38.861630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:38.861630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:38.861630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:38.861630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:38.861630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:38.861630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:46:38.861630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:38.861630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:38.861630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:38.861630 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:38.861984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:38.861984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:38.861984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:38.861984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:38.861984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:38.861984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:38.861984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:38.861984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:38.861984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:38.861984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:38.861984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:38.861984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:38.861984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:38.861984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:38.861984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:38.861984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:38.861984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:39.037887 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:39.038466 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:39.038466 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:39.041014 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:39.041014 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:39.041014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.041744 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:39.041744 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:39.041744 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:39.041744 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:39.041744 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:39.041744 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:39.041744 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.041744 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:39.043105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.043105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:39.043105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:39.043105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:39.043105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:39.043105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:46:39.043105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:39.043105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:39.043105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:39.043105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:39.043105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:39.043105 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:39.043925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:39.043925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:39.043925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:39.043925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:39.043925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:39.043925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:39.043925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:39.043925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:39.043925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:39.043925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:39.043925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:39.043925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:39.043925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:39.044541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.044541 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:39.045984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.045984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:39.045984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:39.045984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:39.045984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:39.045984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:39.045984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:39.045984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:39.045984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:39.045984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:39.045984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:39.045984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:39.045984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:39.045984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:39.045984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:39.045984 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:39.047026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:46:39.047026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:39.047026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:39.047026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:39.047026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:39.047026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:39.047026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:39.047026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:39.047026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:39.047026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:39.047026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:39.047026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:39.047026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:39.047026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:39.047026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:39.047026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:39.047026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:39.047026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:39.047026 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:39.047542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:39.047542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.047542 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:39.049818 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:39.050096 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:39.050210 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:39.050328 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:46:39.051493 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:39.051493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.051493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:39.051493 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:39.051493 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:39.051493 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:39.051493 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:39.051493 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:39.051493 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.051493 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:39.052299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.052299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:46:39.052299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:39.052299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:39.052299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:39.052299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:39.052299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:39.052299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:39.052299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:39.052299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:39.052299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:39.052299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:46:39.052299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:39.052299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:46:39.052299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:46:39.052750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:39.052750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:46:39.052750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:39.052750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:46:39.052750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:46:39.052750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:39.052750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:46:39.052750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.052750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:46:39.053133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.053133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:39.053133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:39.053133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:39.053133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:39.053133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:46:39.053133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:46:39.053133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:46:39.053133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:39.053133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:39.053133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:39.053133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:39.053414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:39.053414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:39.053414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:39.053414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:39.053414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:39.053414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:39.053414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:39.053414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:39.053414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:39.053414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:39.053414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:39.053414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:39.053414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:39.053414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:39.053723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:39.053723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:39.053723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.053723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:39.054168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.054168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:39.054168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:39.054168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:39.054168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:39.054168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:39.054168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:39.054168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:39.054168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:39.054168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:39.054168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:39.054168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:39.054168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:39.054168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:39.054168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:39.054168 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:39.054494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:46:39.054494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:46:39.054494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:46:39.054494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:39.054494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:39.054494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:39.054494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:39.054494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:39.054494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:39.054494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:39.054494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:39.054494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:39.054494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:39.054494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:39.054494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:39.054494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:39.054494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:39.054494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:39.054890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:39.054890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:39.054890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:39.054890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:39.054890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:39.054890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.054890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:39.070574 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:39.071916 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:39.072098 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:39.072315 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:39.073949 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:39.073949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.073949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:39.073949 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:39.073949 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:39.073949 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:39.073949 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:39.073949 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:39.073949 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.073949 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:39.075847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.075847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:46:39.075847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:39.075847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:39.075847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:39.075847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:39.075847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:39.075847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:39.075847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:39.075847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:39.075847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:39.075847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:46:39.075847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:39.075847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:46:39.075847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:46:39.076473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:39.076473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:46:39.076473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:39.076473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:46:39.076473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:46:39.076473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:39.076473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:46:39.076473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.076473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:46:39.076473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.076473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:39.076473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:39.076473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:39.076473 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:39.077030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:46:39.077030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:46:39.077030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:39.077030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:39.077030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:39.077030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:39.077030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:39.077030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:39.077030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:39.077030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:39.077030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:39.077030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:39.077030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:39.077434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:39.077434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:39.077434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:39.077434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:39.077434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:39.077434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:39.077434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:39.077434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:39.077434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:39.077434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.077434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:39.077434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.078166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:39.078166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:39.078166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:39.078166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:39.078166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:39.078166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:39.078166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:39.078166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:39.078166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:39.078166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:39.078166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:39.078166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:39.078166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:39.078166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:39.078166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:39.078166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:46:39.078166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:46:39.078166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:39.078166 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:39.079569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:39.079569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:39.079569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:39.079569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:39.079569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:39.079569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:39.079569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:39.079569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:39.079569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:39.079569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:39.079569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:39.079569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:39.079569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:39.079569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:39.079569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:39.079569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:39.079569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:39.079569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:39.079569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.079569 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:39.150178 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:39.150555 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:39.150663 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:39.152715 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:39.152715 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:39.152715 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:39.152715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.152715 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:39.152715 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:39.152715 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:39.152715 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:39.152715 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:39.152715 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:39.152715 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.152715 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:39.153548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.153548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:39.153548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:39.153548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:39.153548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:39.153548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:46:39.153548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:39.153548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:39.153548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:39.153548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:39.153548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:39.153548 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:39.153892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:39.153892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:39.153892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:39.153892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:39.153892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:39.153892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:39.153892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:39.153892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:39.153892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:39.153892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:39.153892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:39.153892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:39.153892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:39.154199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.154199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:39.154199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.154199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:39.154199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:39.154199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:39.154199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:39.154199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:39.154199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:39.154199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:39.154199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:39.154199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:39.154199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:39.154199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:39.154199 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:39.154571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:39.154571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:39.154571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:39.154571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:46:39.154571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:39.154571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:39.154571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:39.154571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:39.154571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:39.154571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:39.154571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:39.154571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:39.154571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:39.154571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:39.154571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:39.154571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:39.154571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:39.154571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:39.154571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:39.155030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:39.155030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:39.155030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:39.155030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:39.155030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:39.155030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:43.822238 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:43.823386 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:43.823386 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:43.825100 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:43.864854 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:43.865094 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:43.865739 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:43.865739 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:43.865739 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:43.865739 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:43.865739 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:43.865739 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:43.865739 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:43.865739 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:43.867814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:43.867814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:43.867814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:43.867814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:43.867814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:43.867814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:46:43.867814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:43.867814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:43.867814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:43.867814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:43.867814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:43.867814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:43.868726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:43.868726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:43.868726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:43.868726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:43.868726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:43.868726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:43.868726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:43.868726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:43.868726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:43.868726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:43.868726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:43.868726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:43.868726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:43.869315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:43.869315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:43.869315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:43.869315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:43.869315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:43.869315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:43.869315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:43.869315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:43.869315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:43.869315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:43.869315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:43.869315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:43.869315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:43.869315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:43.869315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:43.870080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:43.870080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:43.870080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:43.870080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:46:43.870080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:43.870080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:43.870080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:43.870080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:43.870080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:43.870080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:43.870080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:43.870080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:43.870080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:43.870080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:43.870080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:43.870080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:43.870080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:43.870080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:43.870080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:43.871245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:43.871245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:43.871245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:43.871245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:43.871245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:43.871245 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:44.038486 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:44.039242 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:44.039486 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:44.040565 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:44.040852 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:44.041035 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.041839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:44.041839 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:44.041839 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:44.041839 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:44.041839 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:44.041839 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:44.041839 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.041839 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:44.043532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.043532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:44.043532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:44.043532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:44.043532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:44.043532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:46:44.043532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:44.043532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:44.043532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:44.043532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:44.043532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:44.043532 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:44.043961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:44.043961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:44.043961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:44.043961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:44.043961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:44.043961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:44.043961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:44.043961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:44.043961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:44.043961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:44.043961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:44.043961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:44.043961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:44.044337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.044337 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:44.045696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.045696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:44.045696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:44.045696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:44.045696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:44.045696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:44.045696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:44.045696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:44.045696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:44.045696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:44.045696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:44.045696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:44.045696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:44.045696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:44.045696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:44.045696 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:44.046418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:46:44.046418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:44.046418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:44.046418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:44.046418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:44.046418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:44.046418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:44.046418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:44.046418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:44.046418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:44.046418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:44.046418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:44.046418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:44.046418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:44.046418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:44.046418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:44.046418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:44.046418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:44.046418 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:44.047281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:44.047281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.047281 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:44.050292 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:44.050647 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:44.050753 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:44.050833 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:46:44.051527 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:44.051766 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.052214 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:44.052214 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:44.052214 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:44.052214 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:44.052214 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:44.052214 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:44.052214 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.052214 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:44.052960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.052960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:46:44.052960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:44.052960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:44.052960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:44.052960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:44.052960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:44.052960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:44.052960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:44.052960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:44.052960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:44.052960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:46:44.052960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:44.052960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:46:44.052960 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:46:44.053637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:44.053637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:46:44.053637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:44.053637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:46:44.053637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:46:44.053637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:44.053637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:46:44.053637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.053637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:46:44.053637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.053637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:44.053637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:44.053637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:44.053637 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:44.054247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:46:44.054247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:46:44.054247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:46:44.054247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:44.054247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:44.054247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:44.054247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:44.054247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:44.054247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:44.054247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:44.054247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:44.054247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:44.054247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:44.054880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:44.054880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:44.054880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:44.054880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:44.054880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:44.054880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:44.054880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:44.054880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:44.054880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:44.054880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:44.054880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.054880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:44.055267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.055267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:44.055267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:44.055267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:44.055267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:44.055267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:44.055267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:44.055267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:44.055267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:44.055267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:44.055267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:44.055267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:44.055267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:44.055267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:44.055267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:44.055267 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:44.055730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:46:44.055730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:46:44.055730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:46:44.055730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:44.055730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:44.055730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:44.055730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:44.055730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:44.055730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:44.055730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:44.055730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:44.055730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:44.055730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:44.055730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:44.055730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:44.055730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:44.055730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:44.055730 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:44.056378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:44.056378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:44.056378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:44.056378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:44.056378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:44.056378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.056378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:44.102953 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:44.102953 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:44.102953 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:44.102953 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:44.102953 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:44.102953 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.104167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:44.104167 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:44.104167 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:44.104167 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:44.104167 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:44.104167 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:44.104167 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.104167 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:44.104167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.104167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:46:44.104167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:44.104167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:44.104167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:44.104167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:44.104167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:44.104167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:44.104167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:44.104167 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:44.106160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:44.106160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:46:44.106160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:44.106160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:46:44.106160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:46:44.106160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:44.106160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:46:44.106160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:44.106160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:46:44.106160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:46:44.106160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:44.106160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:46:44.106160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.106160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:46:44.106160 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.006 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.107178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:44.107178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:44.107178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:44.107178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:44.107178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:46:44.107178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:46:44.107178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:44.107178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:44.107178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:44.107178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:44.107178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:44.107178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:44.107178 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:44.107888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:44.107888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:44.107888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:44.107888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:44.107888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:44.107888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:44.107888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:44.107888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:44.107888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:44.107888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:44.107888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:44.107888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:44.107888 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:44.108528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.108528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:44.108528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.108528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:44.108528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:44.108528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:44.108528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:44.108528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:44.108528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:44.108528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:44.108528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:44.108528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:44.108528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:44.108528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:44.108528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:44.109429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:44.109429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:44.109429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:44.109429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:46:44.109429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:46:44.109429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:44.109429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:44.109429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:44.109429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:44.109429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:44.109429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:44.109429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:44.109429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:44.109429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:44.109429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:44.109429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:44.109429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:44.109429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:44.109429 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:44.110528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:44.110528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:44.110528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:44.110528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:44.110528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:44.110528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:44.110528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.110528 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:44.151009 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:44.151620 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:44.151999 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:44.152268 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:44.152989 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:44.153165 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:44.153338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.153982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:44.153982 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:44.153982 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:44.153982 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:44.153982 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:44.153982 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:44.153982 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.153982 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:44.155378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.155378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:44.155378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:44.155378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:44.155378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:44.155378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:46:44.155378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:44.155378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:44.155378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:44.155378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:44.155378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:44.155378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:44.155720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:44.155720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:44.155720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:44.155720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:44.155720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:44.155720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:44.155720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:44.155720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:44.155720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:44.155720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:44.155720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:44.155720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:44.155720 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:44.156050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.156050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:44.156857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.156857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:44.156857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:44.156857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:44.156857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:44.156857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:44.156857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:44.156857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:44.156857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:44.156857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:44.156857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:44.156857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:44.156857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:44.156857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:44.156857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:44.156857 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:44.157299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:46:44.157299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:44.157299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:44.157299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:44.157299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:44.157299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:44.157299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:44.157299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:44.157299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:44.157299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:44.157299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:44.157299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:44.157299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:44.157299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:44.157299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:44.157299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:44.157299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:44.157299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:44.157299 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:44.157858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:44.157858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:44.157858 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:48.823084 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:48.823983 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:48.823983 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:48.824501 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:48.865933 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:48.866241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:48.866955 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:48.866955 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:48.866955 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:48.866955 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:48.866955 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:48.866955 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:48.866955 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:48.866955 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:48.867913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.043 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:48.867913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:48.867913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:48.867913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:48.867913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:48.867913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:46:48.867913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:48.867913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:48.867913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:48.867913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:48.867913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:48.867913 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:48.868654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:48.868654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:48.868654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:48.868654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:48.868654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:48.868654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:48.868654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:48.868654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:48.868654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:48.868654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:48.868654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:48.868654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:48.868654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:48.869417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:48.869417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:48.869417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:48.869417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:48.869417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:48.869417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:48.869417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:48.869417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:48.869417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:48.869417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:48.869417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:48.869417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:48.869417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:48.869417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:48.869417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:48.870306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:48.870306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:48.870306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:48.870306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:46:48.870306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:48.870306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:48.870306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:48.870306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:48.870306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:48.870306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:48.870306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:48.870306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:48.870306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:48.870306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:48.870306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:48.870306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:48.870306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:48.870306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:48.870306 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:48.871438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:48.871438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:48.871438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:48.871438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:48.871438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:48.871438 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:49.039384 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:49.039840 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:49.039936 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:49.041107 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:49.041926 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:49.041926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.042926 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:49.042926 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:49.042926 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:49.042926 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:49.042926 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:49.042926 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:49.042926 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.042926 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:49.044604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.044604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:49.044604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:49.044604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:49.044604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:49.044604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:46:49.044604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:49.044604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:49.044604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:49.044604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:49.044604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:49.044604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:49.045356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:49.045356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:49.045356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:49.045356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:49.045356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:49.045356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:49.045356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:49.045356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:49.045356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:49.045356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:49.045356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:49.045356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:49.045356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:49.045727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.045727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:49.046348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.046348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:49.046348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:49.046348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:49.046348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:49.046348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:49.046348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:49.046348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:49.046348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:49.046348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:49.046348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:49.046348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:49.046348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:49.046348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:49.046348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:49.046348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:49.046876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:46:49.046876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:49.046876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:49.046876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:49.046876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:49.046876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:49.046876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:49.046876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:49.046876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:49.046876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:49.046876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:49.046876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:49.046876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:49.046876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:49.046876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:49.046876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:49.046876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:49.046876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:49.046876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:49.047382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:49.047382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.047382 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:49.050757 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:49.051038 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:49.051151 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:49.051274 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:46:49.051839 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:49.051951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.052511 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:49.052511 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:49.052511 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:49.052511 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:49.052511 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:49.052511 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:49.052511 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.052511 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:49.053272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.053272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:46:49.053272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:49.053272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:49.053272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:49.053272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:49.053272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:49.053272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:49.053272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:49.053272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:49.053272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:49.053272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:46:49.053272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:49.053272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:46:49.053272 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:46:49.053892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:49.053892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:46:49.053892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:49.053892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:46:49.053892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:46:49.053892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:49.053892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:46:49.053892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.053892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:46:49.054183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.054183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:49.054183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:49.054183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:49.054183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:49.054183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:46:49.054183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:46:49.054183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:46:49.054183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:49.054183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:49.054183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:49.054183 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:49.054571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:49.054571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:49.054571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:49.054571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:49.054571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:49.054571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:49.054571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:49.054571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:49.054571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:49.054571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:49.054571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:49.054571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:49.054571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:49.054571 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:49.054998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:49.054998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:49.054998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.054998 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:49.055136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.055136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:49.055136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:49.055136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:49.055136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:49.055136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:49.055136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:49.055136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:49.055136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:49.055136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:49.055136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:49.055136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:49.055136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:49.055136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:49.055136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:49.055136 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:49.055575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:46:49.055575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:46:49.055575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:46:49.055575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:49.055575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:49.055575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:49.055575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:49.055575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:49.055575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:49.055575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:49.055575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:49.055575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:49.055575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:49.055575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:49.055575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:49.055575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:49.055575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:49.055575 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:49.056081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:49.056081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:49.056081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:49.056081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:49.056081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:49.056081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.056081 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:49.072539 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:49.072921 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:49.073250 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:49.073381 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:49.073989 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:49.074408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.075211 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:49.075211 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:49.075211 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:49.075211 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:49.075211 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:49.075211 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:49.075211 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.075211 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:49.076417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.076417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:46:49.076417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:49.076417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:49.076417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:49.076417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:49.076417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:49.076417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:49.076417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:49.076417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:49.076417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:49.076417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:46:49.076417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:49.076417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:46:49.076417 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:46:49.077205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:49.077205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:46:49.077205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:49.077205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:46:49.077205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:46:49.077205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:49.077205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:46:49.077205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.077205 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:46:49.078076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.078076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:49.078076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:49.078076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:49.078076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:49.078076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:46:49.078076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:46:49.078076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:49.078076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:49.078076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:49.078076 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:49.078603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:49.078603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:49.078603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:49.078603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:49.078603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:49.078603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:49.078603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:49.078603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:49.078603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:49.078603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:49.078603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:49.078603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:49.078603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:49.078603 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:49.079241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:49.079241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:49.079241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.079241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:49.079896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.079896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:49.079896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:49.079896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:49.079896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:49.079896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:49.079896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:49.079896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:49.079896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:49.079896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:49.079896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:49.079896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:49.079896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:49.079896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:49.079896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:49.079896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:49.079896 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:46:49.080841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:46:49.080841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:49.080841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:49.080841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:49.080841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:49.080841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:49.080841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:49.080841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:49.080841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:49.080841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:49.080841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:49.080841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:49.080841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:49.080841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:49.080841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:49.080841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:49.080841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:49.080841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:49.080841 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:49.081654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:49.081654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:49.081654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.081654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:49.151569 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:49.152156 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:49.152350 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:49.152663 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:49.153409 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:49.153836 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:49.154083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.154927 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:49.154927 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:49.154927 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:49.154927 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:49.154927 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:49.154927 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:49.154927 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.154927 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:49.155644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.155644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:49.155644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:49.155644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:49.155644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:49.155644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:46:49.155644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:49.155644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:49.155644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:49.155644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:49.155644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:49.155644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:49.156150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:49.156150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:49.156150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:49.156150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:49.156150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:49.156150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:49.156150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:49.156150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:49.156150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:49.156150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:49.156150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:49.156150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:49.156150 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:49.156560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.156560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:49.156560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.156560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:49.156560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:49.156560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:49.156560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:49.156560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:49.156560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:49.156560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:49.156560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:49.156560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:49.156560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:49.156560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:49.156560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:49.157147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:49.157147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:49.157147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:49.157147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:46:49.157147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:49.157147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:49.157147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:49.157147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:49.157147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:49.157147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:49.157147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:49.157147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:49.157147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:49.157147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:49.157147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:49.157147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:49.157147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:49.157147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:49.157147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:49.157860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:49.157860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:49.157860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:49.157860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:49.157860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:49.157860 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:53.823888 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:53.824628 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:53.824788 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:53.825698 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:53.857662 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:53.857982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:53.858584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:53.858584 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:53.858584 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:53.858584 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:53.858584 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:53.858584 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:53.858584 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:53.858584 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:53.859613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:53.859613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:53.859613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:53.859613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:53.859613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:53.859613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:46:53.859613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:53.859613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:53.859613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:53.859613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:53.859613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:53.859613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:53.860276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:53.860276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:53.860276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:53.860276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:53.860276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:53.860276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:53.860276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:53.860276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:53.860276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:53.860276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:53.860276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:53.860276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:53.860276 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:53.860785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:53.860785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:53.860876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:53.860876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:53.860876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:53.860876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:53.860876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:53.860876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:53.860876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:53.860876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:53.860876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:53.860876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:53.860876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:53.860876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:53.860876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:53.860876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:53.860876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:53.860876 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:53.862158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:46:53.862158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:53.862158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:53.862158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:53.862158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:53.862158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:53.862158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:53.862158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:53.862158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:53.862158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:53.862158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:53.862158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:53.862158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:53.862158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:53.862158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:53.862158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:53.862158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:53.862158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:53.862158 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:53.863760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:53.863760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:53.863760 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:54.040227 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:54.041243 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:54.041505 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:54.042239 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:54.042433 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:54.042584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.043273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:54.043273 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:54.043273 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:54.043273 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:54.043273 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:54.043273 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:54.043273 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.043273 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:54.044338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.044338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:54.044338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:54.044338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:54.044338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:54.044338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:46:54.044338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:54.044338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:54.044338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:54.044338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:54.044338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:54.044338 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:54.045030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:54.045030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:54.045030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:54.045030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:54.045030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:54.045030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:54.045030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:54.045030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:54.045030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:54.045030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:54.045030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:54.045030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:54.045030 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:54.045902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.045902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:54.045902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.045902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:54.045902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:54.045902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:54.045902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:54.045902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:54.045902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:54.045902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:54.045902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:54.045902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:54.045902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:54.045902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:54.045902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:54.046611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:54.046611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:54.046611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:54.046611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:46:54.046611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:54.046611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:54.046611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:54.046611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:54.046611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:54.046611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:54.046611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:54.046611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:54.046611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:54.046611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:54.046611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:54.046611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:54.046611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:54.046611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:54.046611 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:54.047544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:54.047544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:54.047544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:54.047544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:54.047544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.047544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:54.051218 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:54.051653 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:54.052454 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:54.052454 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:46:54.052454 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:54.052454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.052890 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:54.052890 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:54.052890 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:54.052890 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:54.052890 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:54.052890 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:54.052890 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.052890 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:54.053586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.053586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:46:54.053586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:54.053586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:54.053586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:54.053586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:54.053586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:54.053586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:54.053586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:54.053586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:54.053586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:54.053586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:46:54.053586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:54.053586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:46:54.053586 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:46:54.054147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:54.054147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:46:54.054147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:54.054147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:46:54.054147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:46:54.054147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:54.054147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:46:54.054147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.054147 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:46:54.054649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.054649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:54.054649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:54.054649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:54.054649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:54.054649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:46:54.054649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:46:54.054649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:46:54.054649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:54.054649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:54.054649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:54.054649 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:54.055074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:54.055074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:54.055074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:54.055074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:54.055074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:54.055074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:54.055074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:54.055074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:54.055074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:54.055074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:54.055074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:54.055074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:54.055074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:54.055074 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:54.055625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:54.055625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:54.055625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.055625 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:54.055791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.055791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:54.055791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:54.055791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:54.055791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:54.055791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:54.055791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:54.055791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:54.055791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:54.055791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:54.055791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:54.055791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:54.055791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:54.055791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:54.055791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:54.055791 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:54.056313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:46:54.056313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:46:54.056313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:46:54.056313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:54.056313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:54.056313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:54.056313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:54.056313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:54.056313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:54.056313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:54.056313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:54.056313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:54.056313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:54.056313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:54.056313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:54.056313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:54.056313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:54.056313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:54.056830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:54.056830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:54.056830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:54.056830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:54.056830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:54.056830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.056830 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:54.073055 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:54.073417 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:54.073539 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:54.073787 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:54.074116 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:54.074381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.074941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:54.074941 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:54.074941 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:54.074941 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:54.074941 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:54.074941 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:54.074941 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.074941 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:54.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:46:54.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:54.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:54.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:54.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:54.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:54.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:54.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:54.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:54.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:54.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:46:54.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:54.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:46:54.075957 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:46:54.076454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:54.076454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:46:54.076454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:54.076454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:46:54.076454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:46:54.076454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:54.076454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:46:54.076454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.076454 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:46:54.076731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.076731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:54.076731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:54.076731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:54.076731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:54.076731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:46:54.076731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:46:54.076731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:54.076731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:54.076731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:54.076731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:54.077060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:54.077060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:54.077060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:54.077060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:54.077060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:54.077060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:54.077060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:54.077060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:54.077060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:54.077060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:54.077060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:54.077060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:54.077060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:54.077060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:54.077411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:54.077411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:54.077411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.077411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:54.078956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.078956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:54.078956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:54.078956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:54.078956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:54.078956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:54.078956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:54.078956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:54.078956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:54.078956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:54.078956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:54.078956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:54.078956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:54.078956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:54.078956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:54.078956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:54.078956 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:46:54.079466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:46:54.079466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:54.079466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:54.079466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:54.079466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:54.079466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:54.079466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:54.079466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:54.079466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:54.079466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:54.079466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:54.079466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:54.079466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:54.079466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:54.079466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:54.079466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:54.079466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:54.079466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:54.079466 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:54.080068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:54.080068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:54.080068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.080068 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:54.152258 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:54.153018 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:54.153239 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:54.153502 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:54.153910 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:54.154098 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:54.154216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.155194 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:54.155194 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:54.155194 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:54.155194 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:54.155194 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:54.155194 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:54.155194 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.155194 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:54.155794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.155794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:54.155794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:54.155794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:54.155794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:54.155794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:46:54.155794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:54.155794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:54.155794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:54.155794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:54.155794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:54.155794 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:54.156191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:54.156191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:54.156191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:54.156191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:54.156191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:54.156191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:54.156191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:54.156191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:54.156191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:54.156191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:54.156191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:54.156191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:54.156191 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:54.156573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.156573 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:54.157113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.157113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:54.157113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:54.157113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:54.157113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:54.157113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:54.157113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:54.157113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:54.157113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:54.157113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:54.157113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:54.157113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:54.157113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:54.157113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:54.157113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:54.157113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:54.157596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:46:54.157596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:54.157596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:54.157596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:54.157596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:54.157596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:54.157596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:54.157596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:54.157596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:54.157596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:54.157596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:54.157596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:54.157596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:54.157596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:54.157596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:54.157596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:54.157596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:54.157596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:54.157596 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:54.158112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:54.158112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:54.158112 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:58.824816 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:58.825589 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:58.825832 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:58.827195 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:58.861798 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:58.862373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:58.862756 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:58.862756 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:58.862756 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:58.862756 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:58.862756 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:58.862756 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:58.862756 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:58.862756 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:58.865389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:58.865389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:58.865389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:58.865389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:58.865389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:58.865389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:46:58.865389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:58.865389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:58.865389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:58.865389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:58.865389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:58.865389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:58.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:58.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:58.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:58.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:58.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:58.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:58.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:58.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:58.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:58.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:58.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:58.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:58.865793 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:58.866212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:58.866212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:58.866212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:58.866212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:58.866212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:58.866212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:58.866212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:58.866212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:58.866212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:58.866212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:58.866212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:58.866212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:58.866212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:58.866212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:58.866212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:58.866710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:58.866710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:58.866710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:58.866710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:46:58.866710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:58.866710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:58.866710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:58.866710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:58.866710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:58.866710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:58.866710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:58.866710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:58.866710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:58.866710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:58.866710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:58.866710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:58.866710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:58.866710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:58.866710 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:58.867243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:58.867243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:58.867243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:58.867243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:58.867243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:58.867243 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:59.041395 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:59.043441 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:59.043441 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:59.044125 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:59.044382 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:59.044558 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.045839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:59.045839 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:59.045839 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:59.045839 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:59.045839 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:59.045839 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:59.045839 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.045839 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:59.046735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.046735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:59.046735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:59.046735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:59.046735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:59.046735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:46:59.046735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:59.046735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:59.046735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:59.046735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:59.046735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:59.046735 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:59.047129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:59.047129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:59.047129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:59.047129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:59.047129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:59.047129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:59.047129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:59.047129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:59.047129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:59.047129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:59.047129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:59.047129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:59.047129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:59.047485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.047485 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:59.049232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.049232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:59.049232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:59.049232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:59.049232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:59.049232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:59.049232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:59.049232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:59.049232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:59.049232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:59.049232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:59.049232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:59.049232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:59.049232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:59.049232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:59.049232 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:59.049750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:46:59.049750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:59.049750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:59.049750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:59.049750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:59.049750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:59.049750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:59.049750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:59.049750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:59.049750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:59.049750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:59.049750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:59.049750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:59.049750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:59.049750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:59.049750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:59.049750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:59.049750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:59.049750 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:59.050348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:59.050348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.050348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:59.051812 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:59.052167 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:59.052366 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:59.052560 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:46:59.052991 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:59.053270 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.053771 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:59.053771 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:59.053771 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:59.053771 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:59.053771 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:59.053771 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:59.053771 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.053771 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:59.054544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.054544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:46:59.054544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:59.054544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:59.054544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:59.054544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:59.054544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:59.054544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:59.054544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:59.054544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:59.054544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:59.054544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:46:59.054544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:59.054544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:46:59.054544 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:46:59.055141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:59.055141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:46:59.055141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:59.055141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:46:59.055141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:46:59.055141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:59.055141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:46:59.055141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.055141 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:46:59.056361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.056361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:59.056361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:59.056361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:59.056361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:59.056361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:46:59.056361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:46:59.056361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:46:59.056361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:59.056361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:59.056361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:59.056361 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:59.056790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:59.056790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:59.056790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:59.056790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:59.056790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:59.056790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:59.056790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:59.056790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:59.056790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:59.056790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:59.056790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:59.056790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:59.056790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:59.056790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:59.057131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:59.057131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:59.057131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.057131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:59.057131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.057131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:59.057131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:59.057131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:59.057131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:59.057131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:59.057131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:59.057131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:59.057131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:59.057131 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:59.057499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:59.057499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:59.057499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:59.057499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:59.057499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:59.057499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:59.057499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:46:59.057499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:46:59.057499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:46:59.057499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:59.057499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:59.057499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:59.057499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:59.057499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:59.057499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:59.057499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:59.057499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:59.057499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:59.057499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:59.058007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:59.058007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:59.058007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:59.058007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:59.058007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:59.058007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:59.058007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:59.058007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:59.058007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:59.058007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:59.058007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.058007 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:59.073671 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:59.074249 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:59.074557 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:59.074628 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:59.075265 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:59.075613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.076271 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:59.076271 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:59.076271 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:59.076271 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:59.076271 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:59.076271 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:59.076271 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.076271 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:59.077517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.077517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:46:59.077517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:59.077517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:59.077517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:59.077517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:59.077517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:59.077517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:59.077517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:59.077517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:46:59.077517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:59.077517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:46:59.077517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:59.077517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:46:59.077517 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:46:59.077949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:59.077949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:46:59.077949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:59.077949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:46:59.077949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:46:59.077949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:59.077949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:46:59.077949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.077949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:46:59.078905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.078905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:59.078905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:59.078905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:59.078905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:59.078905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:46:59.078905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:46:59.078905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:59.078905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:59.078905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:59.078905 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:59.079257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:59.079257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:59.079257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:59.079257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:59.079257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:59.079257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:59.079257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:59.079257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:59.079257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:59.079257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:59.079257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:59.079257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:59.079257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:59.079257 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:59.079622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:59.079622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:59.079622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.079622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:59.080629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.080629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:59.080629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:59.080629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:59.080629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:59.080629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:59.080629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:59.080629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:59.080629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:59.080629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:59.080629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:59.080629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:59.080629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:59.080629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:59.080629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:59.080629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:59.080629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:46:59.081170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:46:59.081170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:46:59.081170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:46:59.081170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:59.081170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:59.081170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:59.081170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:59.081170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:59.081170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:59.081170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:59.081170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:59.081170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:59.081170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:59.081170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:59.081170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:59.081170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:59.081170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:59.081170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:59.081170 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:59.081700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:59.081700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:59.081700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.081700 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:46:59.153086 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:46:59.153705 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:46:59.153913 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:46:59.154203 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:46:59.154805 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:59.155185 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:46:59.155414 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.156816 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:46:59.156816 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:59.156816 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:46:59.156816 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:46:59.156816 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:59.156816 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:46:59.156816 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.156816 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:46:59.159599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.159599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:46:59.159599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:59.159599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:46:59.159599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:59.159599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:46:59.159599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:59.159599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:59.159599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:59.159599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:59.159599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:59.159599 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:59.160031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:46:59.160031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:59.160031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:46:59.160031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:59.160031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:46:59.160031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:46:59.160031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:59.160031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:46:59.160031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:59.160031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:46:59.160031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:46:59.160031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:59.160031 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:46:59.160387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.160387 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:46:59.160624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.160624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:46:59.160624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:46:59.160624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:46:59.160624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:46:59.160624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:46:59.160624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:59.160624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:46:59.160624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:46:59.160624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:59.160624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:46:59.160624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:46:59.160624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:46:59.160624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:46:59.160624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:46:59.160624 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:46:59.161124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:46:59.161124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:46:59.161124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:59.161124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:59.161124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:46:59.161124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:46:59.161124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:46:59.161124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:46:59.161124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:46:59.161124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:46:59.161124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:46:59.161124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:46:59.161124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:46:59.161124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:46:59.161124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:46:59.161124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:46:59.161124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:46:59.161124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:46:59.161124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:46:59.161669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:46:59.161669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:46:59.161669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:03.825385 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:03.826036 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:03.826198 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:03.827313 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:03.867134 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:03.867535 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:03.868381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:03.868381 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:03.868381 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:03.868381 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:03.868381 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:03.868381 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:03.868381 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:03.868381 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:03.869157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.042 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:03.869157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:03.869157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:03.869157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:03.869157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:03.869157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:47:03.869157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:03.869157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:03.869157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:03.869157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:03.869157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:03.869157 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:03.869560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:03.869560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:03.869560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:03.869560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:03.869560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:03.869560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:03.869560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:03.869560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:03.869560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:03.869560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:03.869560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:03.869560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:03.869560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:03.870050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:03.870050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:03.870050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:03.870050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:03.870050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:03.870050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:03.870050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:03.870050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:03.870050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:03.870050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:03.870050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:03.870050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:03.870050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:03.870050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:03.870050 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:03.870769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:03.870769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:03.870769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:03.870769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:47:03.870769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:03.870769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:03.870769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:03.870769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:03.870769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:03.870769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:03.870769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:03.870769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:03.870769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:03.870769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:03.870769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:03.870769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:03.870769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:03.870769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:03.870769 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:03.871563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:03.871563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:03.871563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:03.871563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:03.871563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:03.871563 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:04.042459 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:04.043040 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:04.043147 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:04.044370 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:04.044716 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:04.045049 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.045917 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:04.045917 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:04.045917 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:04.045917 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:04.045917 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:04.045917 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:04.045917 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.045917 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:04.046949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.046949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:04.046949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:04.046949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:04.046949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:04.046949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:47:04.046949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:04.046949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:04.046949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:04.046949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:04.046949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:04.046949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:04.047282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:04.047282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:04.047282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:04.047282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:04.047282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:04.047282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:04.047282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:04.047282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:04.047282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:04.047282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:04.047282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:04.047282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:04.047282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:04.047640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.047640 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:04.047967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.047967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:04.047967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:04.047967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:04.047967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:04.047967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:04.047967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:04.047967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:04.047967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:04.047967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:04.047967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:04.047967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:04.047967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:04.047967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:04.047967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:04.047967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:04.048723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:47:04.048723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:04.048723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:04.048723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:04.048723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:04.048723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:04.048723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:04.048723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:04.048723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:04.048723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:04.048723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:04.048723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:04.048723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:04.048723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:04.048723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:04.048723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:04.048723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:04.048723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:04.048723 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:04.049463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:04.049463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.049463 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:04.052353 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:04.052647 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:04.052751 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:04.052922 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:47:04.053281 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:04.053484 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.054055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:04.054055 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:04.054055 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:04.054055 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:04.054055 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:04.054055 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:04.054055 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.054055 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:04.054868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.054868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:47:04.054868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:04.054868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:04.054868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:04.054868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:04.054868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:04.054868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:04.054868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:04.054868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:04.054868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:04.054868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:47:04.054868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:04.054868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:47:04.054868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:47:04.055313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:04.055313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:47:04.055313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:04.055313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:47:04.055313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:47:04.055313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:04.055313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:47:04.055313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.055313 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:47:04.055789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.055789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:04.055789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:04.055789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:04.055789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:04.055789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:47:04.055789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:47:04.055789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:47:04.055789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:04.055789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:04.055789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:04.055789 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:04.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:04.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:04.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:04.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:04.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:04.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:04.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:04.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:04.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:04.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:04.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:04.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:04.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:04.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:04.056494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:04.056494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:04.056494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.056494 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:04.056768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.056768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:04.056768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:04.056768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:04.056768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:04.056768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:04.056768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:04.056768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:04.056768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:04.056768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:04.056768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:04.056768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:04.056768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:04.056768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:04.056768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:04.056768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:04.057216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:47:04.057216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:47:04.057216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:47:04.057216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:04.057216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:04.057216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:04.057216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:04.057216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:04.057216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:04.057216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:04.057216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:04.057216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:04.057216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:04.057216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:04.057216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:04.057216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:04.057216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:04.057216 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:04.057722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:04.057722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:04.057722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:04.057722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:04.057722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:04.057722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.057722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:04.074274 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:04.074664 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:04.075071 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:04.075071 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:04.075643 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:04.076075 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.076920 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:04.076920 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:04.076920 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:04.076920 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:04.076920 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:04.076920 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:04.076920 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.076920 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:04.078103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.078103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:47:04.078103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:04.078103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:04.078103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:04.078103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:04.078103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:04.078103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:04.078103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:04.078103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:04.078103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:04.078103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:47:04.078103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:04.078103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:47:04.078103 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:47:04.078514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:04.078514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:47:04.078514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:04.078514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:47:04.078514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:47:04.078514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:04.078514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:47:04.078514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.078514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:47:04.079493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.079493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:04.079493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:04.079493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:04.079493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:04.079493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:47:04.079493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:47:04.079493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:04.079493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:04.079493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:04.079493 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:04.079832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:04.079832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:04.079832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:04.079832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:04.079832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:04.079832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:04.079832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:04.079832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:04.079832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:04.079832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:04.079832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:04.079832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:04.079832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:04.079832 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:04.080377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:04.080377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:04.080377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.080377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:04.081262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.081262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:04.081262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:04.081262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:04.081262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:04.081262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:04.081262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:04.081262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:04.081262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:04.081262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:04.081262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:04.081262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:04.081262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:04.081262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:04.081262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:04.081262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:04.081262 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:47:04.081776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:47:04.081776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:04.081776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:04.081776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:04.081776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:04.081776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:04.081776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:04.081776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:04.081776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:04.081776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:04.081776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:04.081776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:04.081776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:04.081776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:04.081776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:04.081776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:04.081776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:04.081776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:04.081776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:04.082776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:04.082776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:04.082776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.082776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:04.153651 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:04.154093 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:04.154266 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:04.154495 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:04.155141 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:04.155546 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:04.155847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.156669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:04.156669 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:04.156669 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:04.156669 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:04.156669 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:04.156669 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:04.156669 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.156669 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:04.158014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.158014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:04.158014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:04.158014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:04.158014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:04.158014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:47:04.158014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:04.158014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:04.158014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:04.158014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:04.158014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:04.158014 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:04.158546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:04.158546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:04.158546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:04.158546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:04.158546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:04.158546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:04.158546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:04.158546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:04.158546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:04.158546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:04.158546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:04.158546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:04.158546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:04.159153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.159153 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:04.159514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.159514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:04.159514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:04.159514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:04.159514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:04.159514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:04.159514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:04.159514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:04.159514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:04.159514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:04.159514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:04.159514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:04.159514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:04.159514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:04.159514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:04.159514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:04.160282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:47:04.160282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:04.160282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:04.160282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:04.160282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:04.160282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:04.160282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:04.160282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:04.160282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:04.160282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:04.160282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:04.160282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:04.160282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:04.160282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:04.160282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:04.160282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:04.160282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:04.160282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:04.160282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:04.161460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:04.161460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:04.161460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:08.826129 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:08.827073 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:08.827280 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:08.828516 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:08.833891 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG dbcounter [-] [106506] Writing DB stats neutron:SELECT=435 {{(pid=106506) stat_writer /opt/stack/data/venv/lib/python3.12/site-packages/dbcounter.py:115}}
Apr 28 20:47:08.860265 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:08.860697 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:08.861595 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:08.861595 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:08.861595 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:08.861595 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:08.861595 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:08.861595 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:08.861595 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:08.861595 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:08.862339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:08.862339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:08.862339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:08.862339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:08.862339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:08.862339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:47:08.862339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:08.862339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:08.862339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:08.862339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:08.862339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:08.862339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:08.862658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:08.862658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:08.862658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:08.862658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:08.862658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:08.862658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:08.862658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:08.862658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:08.862658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:08.862658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:08.862658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:08.862658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:08.862658 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:08.862989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:08.862989 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:08.863171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:08.863171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:08.863171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:08.863171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:08.863171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:08.863171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:08.863171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:08.863171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:08.863171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:08.863171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:08.863171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:08.863171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:08.863171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:08.863171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:08.863171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:08.863171 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:08.863705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:47:08.863705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:08.863705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:08.863705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:08.863705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:08.863705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:08.863705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:08.863705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:08.863705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:08.863705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:08.863705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:08.863705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:08.863705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:08.863705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:08.863705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:08.863705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:08.863705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:08.863705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:08.863705 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:08.864142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:08.864142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:08.864142 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:09.043167 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:09.043961 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:09.044135 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:09.045372 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:09.045738 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:09.046092 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.046887 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:09.046887 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:09.046887 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:09.046887 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:09.046887 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:09.046887 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:09.046887 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.046887 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:09.048639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.048639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:09.048639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:09.048639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:09.048639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:09.048639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:47:09.048639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:09.048639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:09.048639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:09.048639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:09.048639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:09.048639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:09.049350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:09.049350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:09.049350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:09.049350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:09.049350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:09.049350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:09.049350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:09.049350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:09.049350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:09.049350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:09.049350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:09.049350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:09.049350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:09.049615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.049615 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:09.050827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.050827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:09.050827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:09.050827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:09.050827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:09.050827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:09.050827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:09.050827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:09.050827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:09.050827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:09.050827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:09.050827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:09.050827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:09.050827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:09.050827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:09.050827 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:09.051182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:47:09.051182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:09.051182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:09.051182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:09.051182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:09.051182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:09.051182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:09.051182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:09.051182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:09.051182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:09.051182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:09.051182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:09.051182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:09.051182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:09.051182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:09.051182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:09.051182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:09.051182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:09.051182 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:09.051604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:09.051604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.051604 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:09.052798 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:09.053105 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:09.053250 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:09.053408 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:47:09.053893 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:09.054180 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.054938 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:09.054938 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:09.054938 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:09.054938 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:09.054938 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:09.054938 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:09.054938 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.054938 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:09.055749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.055749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:47:09.055749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:09.055749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:09.055749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:09.055749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:09.055749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:09.055749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:09.055749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:09.055749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:09.055749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:09.055749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:47:09.055749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:09.055749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:47:09.055749 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:47:09.056331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:09.056331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:47:09.056331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:09.056331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:47:09.056331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:47:09.056331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:09.056331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:47:09.056331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.056331 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:47:09.056856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.056856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:09.056856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:09.056856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:09.056856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:09.056856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:47:09.056856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:47:09.056856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:47:09.056856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:09.056856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:09.056856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:09.056856 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:09.057360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:09.057360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:09.057360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:09.057360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:09.057360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:09.057360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:09.057360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:09.057360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:09.057360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:09.057360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:09.057360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:09.057360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:09.057360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:09.057360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:09.057780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:09.057780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:09.057780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.057780 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:09.058124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.058124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:09.058124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:09.058124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:09.058124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:09.058124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:09.058124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:09.058124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:09.058124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:09.058124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:09.058124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:09.058124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:09.058124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:09.058124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:09.058124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:09.058124 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:09.058628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:47:09.058628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:47:09.058628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:47:09.058628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:09.058628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:09.058628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:09.058628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:09.058628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:09.058628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:09.058628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:09.058628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:09.058628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:09.058628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:09.058628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:09.058628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:09.058628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:09.058628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:09.058628 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:09.059315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:09.059315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:09.059315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:09.059315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:09.059315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:09.059315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.059315 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:09.074832 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:09.075226 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:09.075339 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:09.075570 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:09.076350 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:09.076556 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.077265 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:09.077265 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:09.077265 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:09.077265 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:09.077265 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:09.077265 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:09.077265 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.077265 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:09.078322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.078322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:47:09.078322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:09.078322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:09.078322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:09.078322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:09.078322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:09.078322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:09.078322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:09.078322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:09.078322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:09.078322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:47:09.078322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:09.078322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:47:09.078322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:47:09.078776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:09.078776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:47:09.078776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:09.078776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:47:09.078776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:47:09.078776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:09.078776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:47:09.078776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.078776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:47:09.080083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.080083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:09.080083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:09.080083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:09.080083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:09.080083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:47:09.080083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:47:09.080083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:09.080083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:09.080083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:09.080083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:09.080411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:09.080411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:09.080411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:09.080411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:09.080411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:09.080411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:09.080411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:09.080411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:09.080411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:09.080411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:09.080411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:09.080411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:09.080411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:09.080411 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:09.080824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:09.080824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:09.080824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.080824 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:09.082070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.082070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:09.082070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:09.082070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:09.082070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:09.082070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:09.082070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:09.082070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:09.082070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:09.082070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:09.082070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:09.082070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:09.082070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:09.082070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:09.082070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:09.082070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:09.082070 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:47:09.082833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:47:09.082833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:09.082833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:09.082833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:09.082833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:09.082833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:09.082833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:09.082833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:09.082833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:09.082833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:09.082833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:09.082833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:09.082833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:09.082833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:09.082833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:09.082833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:09.082833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:09.082833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:09.082833 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:09.083347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:09.083347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:09.083347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.083347 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:09.154266 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:09.154822 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:09.155031 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:09.155304 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:09.155868 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:09.156229 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:09.156443 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.157287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:09.157287 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:09.157287 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:09.157287 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:09.157287 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:09.157287 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:09.157287 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.157287 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:09.158716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.158716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:09.158716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:09.158716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:09.158716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:09.158716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:47:09.158716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:09.158716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:09.158716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:09.158716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:09.158716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:09.158716 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:09.159560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:09.159560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:09.159560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:09.159560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:09.159560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:09.159560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:09.159560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:09.159560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:09.159560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:09.159560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:09.159560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:09.159560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:09.159560 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:09.160373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.160373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:09.160373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.160373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:09.160373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:09.160373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:09.160373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:09.160373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:09.160373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:09.160373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:09.160373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:09.160373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:09.160373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:09.160373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:09.160373 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:09.161475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:09.161475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:09.161475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:09.161475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:47:09.161475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:09.161475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:09.161475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:09.161475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:09.161475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:09.161475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:09.161475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:09.161475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:09.161475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:09.161475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:09.161475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:09.161475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:09.161475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:09.161475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:09.161475 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:09.162731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:09.162731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:09.162731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:09.162731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:09.162731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:09.162731 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:13.827151 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:13.830136 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:13.830340 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:13.831171 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:13.873794 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:13.874192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:13.875201 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:13.875201 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:13.875201 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:13.875201 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:13.875201 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:13.875201 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:13.875201 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:13.875201 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:13.876738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.045 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:13.876738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:13.876738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:13.876738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:13.876738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:13.876738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:47:13.876738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:13.876738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:13.876738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:13.876738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:13.876738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:13.876738 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:13.877055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:13.877055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:13.877055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:13.877055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:13.877055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:13.877055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:13.877055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:13.877055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:13.877055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:13.877055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:13.877055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:13.877055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:13.877055 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:13.877349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:13.877349 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:13.878965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:13.878965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:13.878965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:13.878965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:13.878965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:13.878965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:13.878965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:13.878965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:13.878965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:13.878965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:13.878965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:13.878965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:13.878965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:13.878965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:13.878965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:13.878965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:13.879350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:47:13.879350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:13.879350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:13.879350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:13.879350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:13.879350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:13.879350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:13.879350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:13.879350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:13.879350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:13.879350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:13.879350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:13.879350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:13.879350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:13.879350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:13.879350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:13.879350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:13.879350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:13.879350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:13.879737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:13.879737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:13.879737 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:14.043808 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:14.044294 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:14.044481 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:14.045557 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:14.045833 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:14.045991 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.046785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:14.046785 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:14.046785 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:14.046785 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:14.046785 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:14.046785 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:14.046785 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.046785 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:14.048419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.048419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:14.048419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:14.048419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:14.048419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:14.048419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:47:14.048419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:14.048419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:14.048419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:14.048419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:14.048419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:14.048419 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:14.048784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:14.048784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:14.048784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:14.048784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:14.048784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:14.048784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:14.048784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:14.048784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:14.048784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:14.048784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:14.048784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:14.048784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:14.048784 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:14.049110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.049110 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:14.050384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.050384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:14.050384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:14.050384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:14.050384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:14.050384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:14.050384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:14.050384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:14.050384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:14.050384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:14.050384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:14.050384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:14.050384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:14.050384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:14.050384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:14.050384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:14.050886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:47:14.050886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:14.050886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:14.050886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:14.050886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:14.050886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:14.050886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:14.050886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:14.050886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:14.050886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:14.050886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:14.050886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:14.050886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:14.050886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:14.050886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:14.050886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:14.050886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:14.050886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:14.050886 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:14.051399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:14.051399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.051399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:14.053123 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:14.053373 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:14.053531 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:14.053660 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:47:14.054086 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:14.054282 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.054808 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:14.054808 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:14.054808 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:14.054808 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:14.054808 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:14.054808 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:14.054808 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.054808 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:14.055584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.055584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:47:14.055584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:14.055584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:14.055584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:14.055584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:14.055584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:14.055584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:14.055584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:14.055584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:14.055584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:14.055584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:47:14.055584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:14.055584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:47:14.055584 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:47:14.056287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:14.056287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:47:14.056287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:14.056287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:47:14.056287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:47:14.056287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:14.056287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:47:14.056287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.056287 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:47:14.056778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.056778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:14.056778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:14.056778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:14.056778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:14.056778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:47:14.056778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:47:14.056778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:47:14.056778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:14.056778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:14.056778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:14.056778 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:14.057083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:14.057083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:14.057083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:14.057083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:14.057083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:14.057083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:14.057083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:14.057083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:14.057083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:14.057083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:14.057083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:14.057083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:14.057083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:14.057083 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:14.057413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:14.057413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:14.057413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.057413 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:14.057844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.057844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:14.057844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:14.057844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:14.057844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:14.057844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:14.057844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:14.057844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:14.057844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:14.057844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:14.057844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:14.057844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:14.057844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:14.057844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:14.057844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:14.057844 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:14.059107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:47:14.059107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:47:14.059107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:47:14.059107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:14.059107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:14.059107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:14.059107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:14.059107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:14.059107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:14.059107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:14.059107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:14.059107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:14.059107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:14.059107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:14.059107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:14.059107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:14.059107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:14.059107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:14.059853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:14.059853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:14.059853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:14.059853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:14.059853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:14.059853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.059853 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:14.075255 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:14.075630 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:14.075998 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:14.076233 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:14.077037 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:14.077717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.078718 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:14.078718 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:14.078718 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:14.078718 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:14.078718 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:14.078718 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:14.078718 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.078718 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:14.080399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.080399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:47:14.080399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:14.080399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:14.080399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:14.080399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:14.080399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:14.080399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:14.080399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:14.080399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:14.080399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:14.080399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:47:14.080399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:14.080399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:47:14.080399 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:47:14.080877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:14.080877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:47:14.080877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:14.080877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:47:14.080877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:47:14.080877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:14.080877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:47:14.080877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.080877 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:47:14.082108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.082108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:14.082108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:14.082108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:14.082108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:14.082108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:47:14.082108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:47:14.082108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:14.082108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:14.082108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:14.082108 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:14.082521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:14.082521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:14.082521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:14.082521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:14.082521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:14.082521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:14.082521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:14.082521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:14.082521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:14.082521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:14.082521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:14.082521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:14.082521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:14.082521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:14.083019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:14.083019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:14.083019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.083019 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:14.084390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.084390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:14.084390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:14.084390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:14.084390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:14.084390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:14.084390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:14.084390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:14.084390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:14.084390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:14.084390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:14.084390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:14.084390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:14.084390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:14.084390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:14.084390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:14.084390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:47:14.085644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:47:14.085644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:14.085644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:14.085644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:14.085644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:14.085644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:14.085644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:14.085644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:14.085644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:14.085644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:14.085644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:14.085644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:14.085644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:14.085644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:14.085644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:14.085644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:14.085644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:14.085644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:14.085644 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:14.086933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:14.086933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:14.086933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.086933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:14.154855 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:14.155333 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:14.155517 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:14.155852 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:14.156623 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:14.157138 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:14.157441 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.158409 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:14.158409 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:14.158409 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:14.158409 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:14.158409 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:14.158409 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:14.158409 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.158409 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:14.159966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.159966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:14.159966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:14.159966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:14.159966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:14.159966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:47:14.159966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:14.159966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:14.159966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:14.159966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:14.159966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:14.159966 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:14.160546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:14.160546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:14.160546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:14.160546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:14.160546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:14.160546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:14.160546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:14.160546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:14.160546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:14.160546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:14.160546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:14.160546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:14.160546 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:14.161790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.161790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:14.161790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.161790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:14.161790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:14.161790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:14.161790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:14.161790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:14.161790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:14.161790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:14.161790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:14.161790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:14.161790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:14.161790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:14.161790 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:14.163252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:14.163252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:14.163252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:14.163252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:47:14.163252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:14.163252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:14.163252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:14.163252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:14.163252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:14.163252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:14.163252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:14.163252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:14.163252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:14.163252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:14.163252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:14.163252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:14.163252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:14.163252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:14.163252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:14.165240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:14.165240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:14.165240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:14.165240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:14.165240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:14.165240 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:18.830371 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:18.831342 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:18.831342 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:18.831504 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:18.860884 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:18.861202 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:18.861873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:18.861873 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:18.861873 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:18.861873 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:18.861873 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:18.861873 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:18.861873 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:18.861873 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:18.862619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.031 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:18.862619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:18.862619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:18.862619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:18.862619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:18.862619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:47:18.862619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:18.862619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:18.862619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:18.862619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:18.862619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:18.862619 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:18.863231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:18.863231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:18.863231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:18.863231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:18.863231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:18.863231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:18.863231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:18.863231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:18.863231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:18.863231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:18.863231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:18.863231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:18.863231 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:18.863880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:18.863880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:18.863880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:18.863880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:18.863880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:18.863880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:18.863880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:18.863880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:18.863880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:18.863880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:18.863880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:18.863880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:18.863880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:18.863880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:18.863880 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:18.864583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:18.864583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:18.864583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:18.864583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:47:18.864583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:18.864583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:18.864583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:18.864583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:18.864583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:18.864583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:18.864583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:18.864583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:18.864583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:18.864583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:18.864583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:18.864583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:18.864583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:18.864583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:18.864583 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:18.866840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:18.866840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:18.866840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:18.866840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:18.866840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:18.866840 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:19.044657 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:19.045121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:19.045121 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:19.045878 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:19.046089 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:19.046273 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.046849 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:19.046849 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:19.046849 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:19.046849 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:19.046849 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:19.046849 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:19.046849 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.046849 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:19.048025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.048025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:19.048025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:19.048025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:19.048025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:19.048025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:47:19.048025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:19.048025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:19.048025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:19.048025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:19.048025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:19.048025 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:19.049513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:19.049513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:19.049513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:19.049513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:19.049513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:19.049513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:19.049513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:19.049513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:19.049513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:19.049513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:19.049513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:19.049513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:19.049513 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:19.050263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.050263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:19.050263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.050263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:19.050263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:19.050263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:19.050263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:19.050263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:19.050263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:19.050263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:19.050263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:19.050263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:19.050263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:19.050263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:19.050263 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:19.051132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:19.051132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:19.051132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:19.051132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:47:19.051132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:19.051132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:19.051132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:19.051132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:19.051132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:19.051132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:19.051132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:19.051132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:19.051132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:19.051132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:19.051132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:19.051132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:19.051132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:19.051132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:19.051132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:19.051910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:19.051910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:19.051910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:19.051910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:19.051910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.051910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:19.053552 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:19.053751 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:19.053816 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:19.053910 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:47:19.054233 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:19.054432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.054943 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:19.054943 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:19.054943 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:19.054943 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:19.054943 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:19.054943 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:19.054943 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.054943 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:19.055462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.055462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:47:19.055462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:19.055462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:19.055462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:19.055462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:19.055462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:19.055462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:19.055462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:19.055462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:19.055462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:19.055462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:47:19.055462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:19.055462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:47:19.055462 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:47:19.055855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:19.055855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:47:19.055855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:19.055855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:47:19.055855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:47:19.055855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:19.055855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:47:19.055855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.055855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:47:19.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:19.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:19.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:19.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:19.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:47:19.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:47:19.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:47:19.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:19.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:19.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:19.056117 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:19.056448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:19.056448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:19.056448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:19.056448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:19.056448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:19.056448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:19.056448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:19.056448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:19.056448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:19.056448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:19.056448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:19.056448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:19.056448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:19.056448 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:19.056893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:19.056893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:19.056893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.056893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:19.056893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.056893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:19.056893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:19.056893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:19.056893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:19.056893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:19.056893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:19.056893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:19.056893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:19.056893 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:19.057332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:19.057332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:19.057332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:19.057332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:19.057332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:19.057332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:19.057332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:47:19.057332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:47:19.057332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:47:19.057332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:19.057332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:19.057332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:19.057332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:19.057332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:19.057332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:19.057332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:19.057332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:19.057332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:19.057332 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:19.057867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:19.057867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:19.057867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:19.057867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:19.057867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:19.057867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:19.057867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:19.057867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:19.057867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:19.057867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:19.057867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.057867 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:19.075788 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:19.076036 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:19.076094 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:19.076199 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:19.076813 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:19.077189 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.077891 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:19.077891 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:19.077891 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:19.077891 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:19.077891 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:19.077891 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:19.077891 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.077891 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:19.078421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.078421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:47:19.078421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:19.078421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:19.078421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:19.078421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:19.078421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:19.078421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:19.078421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:19.078421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:19.078421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:19.078421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:47:19.078421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:19.078421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:47:19.078421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:47:19.078809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:19.078809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:47:19.078809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:19.078809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:47:19.078809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:47:19.078809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:19.078809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:47:19.078809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.078809 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:47:19.079046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.079046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:19.079046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:19.079046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:19.079046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:19.079046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:47:19.079046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:47:19.079046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:19.079046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:19.079046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:19.079046 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:19.079339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:19.079339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:19.079339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:19.079339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:19.079339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:19.079339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:19.079339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:19.079339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:19.079339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:19.079339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:19.079339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:19.079339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:19.079339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:19.079339 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:19.079726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:19.079726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:19.079726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.079726 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:19.079839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.079839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:19.079839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:19.079839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:19.079839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:19.079839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:19.079839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:19.079839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:19.079839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:19.079839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:19.079839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:19.079839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:19.079839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:19.079839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:19.079839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:19.079839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:19.079839 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:47:19.080322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:47:19.080322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:19.080322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:19.080322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:19.080322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:19.080322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:19.080322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:19.080322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:19.080322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:19.080322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:19.080322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:19.080322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:19.080322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:19.080322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:19.080322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:19.080322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:19.080322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:19.080322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:19.080322 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:19.080967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:19.080967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:19.080967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.080967 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:19.155362 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:19.155791 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:19.155888 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:19.156040 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:19.156557 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:19.157045 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:19.157288 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.158018 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:19.158018 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:19.158018 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:19.158018 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:19.158018 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:19.158018 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:19.158018 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.158018 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:19.158752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.158752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:19.158752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:19.158752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:19.158752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:19.158752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:47:19.158752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:19.158752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:19.158752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:19.158752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:19.158752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:19.158752 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:19.159301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:19.159301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:19.159301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:19.159301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:19.159301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:19.159301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:19.159301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:19.159301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:19.159301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:19.159301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:19.159301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:19.159301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:19.159301 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:19.159874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.159874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:19.159874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.159874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:19.159874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:19.159874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:19.159874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:19.159874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:19.159874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:19.159874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:19.159874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:19.159874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:19.159874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:19.159874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:19.159874 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:19.160538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:19.160538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:19.160538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:19.160538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:47:19.160538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:19.160538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:19.160538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:19.160538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:19.160538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:19.160538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:19.160538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:19.160538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:19.160538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:19.160538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:19.160538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:19.160538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:19.160538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:19.160538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:19.160538 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:19.161381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:19.161381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:19.161381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:19.161381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:19.161381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:19.161381 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:23.830870 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:23.831714 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:23.831714 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:23.832608 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:23.868164 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:23.868401 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:23.869129 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:23.869129 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:23.869129 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:23.869129 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:23.869129 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:23.869129 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:23.869129 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:23.869129 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:23.869965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.038 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:23.869965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:23.869965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:23.869965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:23.869965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:23.869965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:47:23.869965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:23.869965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:23.869965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:23.869965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:23.869965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:23.869965 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:23.870740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:23.870740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:23.870740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:23.870740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:23.870740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:23.870740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:23.870740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:23.870740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:23.870740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:23.870740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:23.870740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:23.870740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:23.870740 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:23.871398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:23.871398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:23.871398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:23.871398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:23.871398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:23.871398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:23.871398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:23.871398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:23.871398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:23.871398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:23.871398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:23.871398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:23.871398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:23.871398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:23.871398 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:23.872230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:23.872230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:23.872230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:23.872230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:47:23.872230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:23.872230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:23.872230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:23.872230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:23.872230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:23.872230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:23.872230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:23.872230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:23.872230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:23.872230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:23.872230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:23.872230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:23.872230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:23.872230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:23.872230 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:23.873259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:23.873259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:23.873259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:23.873259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:23.873259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:23.873259 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:24.045217 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:24.046115 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:24.046115 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:24.047553 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:24.047887 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:24.048113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.048977 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:24.048977 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:24.048977 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:24.048977 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:24.048977 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:24.048977 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:24.048977 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.048977 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:24.050592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.050592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:24.050592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:24.050592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:24.050592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:24.050592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:47:24.050592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:24.050592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:24.050592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:24.050592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:24.050592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:24.050592 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:24.051327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:24.051327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:24.051327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:24.051327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:24.051327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:24.051327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:24.051327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:24.051327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:24.051327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:24.051327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:24.051327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:24.051327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:24.051327 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:24.052042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.052042 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:24.052578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.052578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:24.052578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:24.052578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:24.052578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:24.052578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:24.052578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:24.052578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:24.052578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:24.052578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:24.052578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:24.052578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:24.052578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:24.052578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:24.052578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:24.052578 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:24.053459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:47:24.053459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:24.053459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:24.053459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:24.053459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:24.053459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:24.053459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:24.053459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:24.053459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:24.053459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:24.053459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:24.053459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:24.053459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:24.053459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:24.053459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:24.053459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:24.053459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:24.053459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:24.053459 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:24.054707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:24.054707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.054707 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:24.054707 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:24.054707 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:24.054707 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:24.054707 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:47:24.055179 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:24.055179 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.055502 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:24.055502 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:24.055502 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:24.055502 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:24.055502 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:24.055502 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:24.055502 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.055502 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:24.056241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.056241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:47:24.056241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:24.056241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:24.056241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:24.056241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:24.056241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:24.056241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:24.056241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:24.056241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:24.056241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:24.056241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:47:24.056241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:24.056241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:47:24.056241 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:47:24.057132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:24.057132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:47:24.057132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:24.057132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:47:24.057132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:47:24.057132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:24.057132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:47:24.057132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.057132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:47:24.057132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.057132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:24.057132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:24.057132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:24.057132 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:24.057521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:47:24.057521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:47:24.057521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:47:24.057521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:24.057521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:24.057521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:24.057521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:24.057521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:24.057521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:24.057521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:24.057521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:24.057521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:24.057521 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:24.058060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:24.058060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:24.058060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:24.058060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:24.058060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:24.058060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:24.058060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:24.058060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:24.058060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:24.058060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:24.058060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.058060 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:24.058499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.058499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:24.058499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:24.058499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:24.058499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:24.058499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:24.058499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:24.058499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:24.058499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:24.058499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:24.058499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:24.058499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:24.058499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:24.058499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:24.058499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:24.058499 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:24.059496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:47:24.059496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:47:24.059496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:47:24.059496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:24.059496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:24.059496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:24.059496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:24.059496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:24.059496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:24.059496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:24.059496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:24.059496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:24.059496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:24.059496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:24.059496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:24.059496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:24.059496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:24.059496 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:24.060613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:24.060613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:24.060613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:24.060613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:24.060613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:24.060613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.060613 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:24.076320 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:24.076705 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:24.076804 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:24.076946 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:24.077373 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:24.077746 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.078461 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:24.078461 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:24.078461 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:24.078461 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:24.078461 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:24.078461 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:24.078461 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.078461 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:24.079233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.079233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:47:24.079233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:24.079233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:24.079233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:24.079233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:24.079233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:24.079233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:24.079233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:24.079233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:24.079233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:24.079233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:47:24.079233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:24.079233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:47:24.079233 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:47:24.080627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:24.080627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:47:24.080627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:24.080627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:47:24.080627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:47:24.080627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:24.080627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:47:24.080627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.080627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:47:24.080627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.080627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:24.080627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:24.080627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:24.080627 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:24.081378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:47:24.081378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:47:24.081378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:24.081378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:24.081378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:24.081378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:24.081378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:24.081378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:24.081378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:24.081378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:24.081378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:24.081378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:24.081378 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:24.081979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:24.081979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:24.081979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:24.081979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:24.081979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:24.081979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:24.081979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:24.081979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:24.081979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:24.081979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.081979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:24.081979 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.082551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:24.082551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:24.082551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:24.082551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:24.082551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:24.082551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:24.082551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:24.082551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:24.082551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:24.082551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:24.082551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:24.082551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:24.082551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:24.082551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:24.082551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:24.082551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:47:24.082551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:47:24.082551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:24.082551 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:24.083936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:24.083936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:24.083936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:24.083936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:24.083936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:24.083936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:24.083936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:24.083936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:24.083936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:24.083936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:24.083936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:24.083936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:24.083936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:24.083936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:24.083936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:24.083936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:24.083936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:24.083936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:24.083936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.083936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:24.156131 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:24.156805 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:24.157059 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:24.157380 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:24.158007 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:24.158341 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:24.158557 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.159389 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:24.159389 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:24.159389 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:24.159389 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:24.159389 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:24.159389 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:24.159389 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.159389 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:24.161087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.161087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:24.161087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:24.161087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:24.161087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:24.161087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:47:24.161087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:24.161087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:24.161087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:24.161087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:24.161087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:24.161087 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:24.162000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:24.162000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:24.162000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:24.162000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:24.162000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:24.162000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:24.162000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:24.162000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:24.162000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:24.162000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:24.162000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:24.162000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:24.162000 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:24.162843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.162843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:24.163343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.163343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:24.163343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:24.163343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:24.163343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:24.163343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:24.163343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:24.163343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:24.163343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:24.163343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:24.163343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:24.163343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:24.163343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:24.163343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:24.163343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:24.163343 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:24.164390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:47:24.164390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:24.164390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:24.164390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:24.164390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:24.164390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:24.164390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:24.164390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:24.164390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:24.164390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:24.164390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:24.164390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:24.164390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:24.164390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:24.164390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:24.164390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:24.164390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:24.164390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:24.164390 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:24.164855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:24.164855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:24.164855 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:28.831377 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:28.832115 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:28.832115 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:28.833425 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:28.868023 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:28.868395 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:28.869254 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:28.869254 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:28.869254 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:28.869254 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:28.869254 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:28.869254 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:28.869254 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:28.869254 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.037 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:47:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:28.870396 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:28.870964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:28.870964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:28.870964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:28.870964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:28.870964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:28.870964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:28.870964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:28.870964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:28.870964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:28.870964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:28.870964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:28.870964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:28.870964 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:28.871310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:28.871310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:28.871310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:28.871310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:28.871310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:28.871310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:28.871310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:28.871310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:28.871310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:28.871310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:28.871310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:28.871310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:28.871310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:28.871310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:28.871310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:28.871792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:28.871792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:28.871792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:28.871792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:47:28.871792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:28.871792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:28.871792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:28.871792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:28.871792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:28.871792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:28.871792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:28.871792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:28.871792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:28.871792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:28.871792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:28.871792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:28.871792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:28.871792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:28.871792 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:28.873585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:28.873585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:28.873585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:28.873585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:28.873585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:28.873585 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:29.045838 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:29.046306 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:29.046569 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:29.047374 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:29.047713 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:29.047869 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.048754 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:29.048754 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:29.048754 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:29.048754 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:29.048754 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:29.048754 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:29.048754 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.048754 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:29.050377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.050377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:29.050377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:29.050377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:29.050377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:29.050377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:47:29.050377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:29.050377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:29.050377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:29.050377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:29.050377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:29.050377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:29.050743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:29.050743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:29.050743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:29.050743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:29.050743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:29.050743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:29.050743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:29.050743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:29.050743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:29.050743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:29.050743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:29.050743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:29.050743 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:29.051407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.051407 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:29.052128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.052128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:29.052128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:29.052128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:29.052128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:29.052128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:29.052128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:29.052128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:29.052128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:29.052128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:29.052128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:29.052128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:29.052128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:29.052128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:29.052128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:29.052128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:29.052622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:47:29.052622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:29.052622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:29.052622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:29.052622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:29.052622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:29.052622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:29.052622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:29.052622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:29.052622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:29.052622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:29.052622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:29.052622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:29.052622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:29.052622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:29.052622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:29.052622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:29.052622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:29.052622 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:29.053344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:29.053344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.053344 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:29.054380 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:29.054824 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:29.055042 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:29.055733 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:47:29.055733 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:29.055962 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.056727 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:29.056727 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:29.056727 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:29.056727 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:29.056727 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:29.056727 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:29.056727 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.056727 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:29.057469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.057469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:47:29.057469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:29.057469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:29.057469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:29.057469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:29.057469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:29.057469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:29.057469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:29.057469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:29.057469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:29.057469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:47:29.057469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:29.057469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:47:29.057469 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:47:29.057941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:29.057941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:47:29.057941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:29.057941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:47:29.057941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:47:29.057941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:29.057941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:47:29.057941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.057941 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:47:29.058308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.058308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:29.058308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:29.058308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:29.058308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:29.058308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:47:29.058308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:47:29.058308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:47:29.058308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:29.058308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:29.058308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:29.058308 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:29.058650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:29.058650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:29.058650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:29.058650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:29.058650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:29.058650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:29.058650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:29.058650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:29.058650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:29.058650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:29.058650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:29.058650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:29.058650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:29.058650 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:29.059348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:29.059348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:29.059348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.059348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:29.059348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.059348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:29.059348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:29.059348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:29.059348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:29.059348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:29.059348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:29.059348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:29.059348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:29.059348 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:29.059768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:29.059768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:29.059768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:29.059768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:29.059768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:29.059768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:29.059768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:47:29.059768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:47:29.059768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:47:29.059768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:29.059768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:29.059768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:29.059768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:29.059768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:29.059768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:29.059768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:29.059768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:29.059768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:29.059768 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:29.060310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:29.060310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:29.060310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:29.060310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:29.060310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:29.060310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:29.060310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:29.060310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:29.060310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:29.060310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:29.060310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.060310 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:29.076720 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:29.077096 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:29.077323 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:29.077531 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:29.078151 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:29.078445 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.079118 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:29.079118 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:29.079118 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:29.079118 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:29.079118 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:29.079118 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:29.079118 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.079118 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:29.080405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.080405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:47:29.080405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:29.080405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:29.080405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:29.080405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:29.080405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:29.080405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:29.080405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:29.080405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:29.080405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:29.080405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:47:29.080405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:29.080405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:47:29.080405 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:47:29.081163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:29.081163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:47:29.081163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:29.081163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:47:29.081163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:47:29.081163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:29.081163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:47:29.081163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.081163 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:47:29.082193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.082193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:29.082193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:29.082193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:29.082193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:29.082193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:47:29.082193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:47:29.082193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:29.082193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:29.082193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:29.082193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:29.082785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:29.082785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:29.082785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:29.082785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:29.082785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:29.082785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:29.082785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:29.082785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:29.082785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:29.082785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:29.082785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:29.082785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:29.082785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:29.082785 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:29.083432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:29.083432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:29.083432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.083432 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:29.084091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.084091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:29.084091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:29.084091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:29.084091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:29.084091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:29.084091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:29.084091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:29.084091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:29.084091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:29.084091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:29.084091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:29.084091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:29.084091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:29.084091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:29.084091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:29.084091 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:47:29.085012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:47:29.085012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:29.085012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:29.085012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:29.085012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:29.085012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:29.085012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:29.085012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:29.085012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:29.085012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:29.085012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:29.085012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:29.085012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:29.085012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:29.085012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:29.085012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:29.085012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:29.085012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:29.085012 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:29.085936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:29.085936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:29.085936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.085936 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:29.156815 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:29.157313 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:29.157513 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:29.157835 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:29.158437 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:29.158874 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:29.159109 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.159999 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:29.159999 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:29.159999 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:29.159999 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:29.159999 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:29.159999 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:29.159999 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.159999 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:29.161439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.161439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:29.161439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:29.161439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:29.161439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:29.161439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:47:29.161439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:29.161439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:29.161439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:29.161439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:29.161439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:29.161439 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:29.162128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:29.162128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:29.162128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:29.162128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:29.162128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:29.162128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:29.162128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:29.162128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:29.162128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:29.162128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:29.162128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:29.162128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:29.162128 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:29.162736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.162736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:29.163235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.163235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:29.163235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:29.163235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:29.163235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:29.163235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:29.163235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:29.163235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:29.163235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:29.163235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:29.163235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:29.163235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:29.163235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:29.163235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:29.163235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:29.163235 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:29.164029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:47:29.164029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:29.164029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:29.164029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:29.164029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:29.164029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:29.164029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:29.164029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:29.164029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:29.164029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:29.164029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:29.164029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:29.164029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:29.164029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:29.164029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:29.164029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:29.164029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:29.164029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:29.164029 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:29.164925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:29.164925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:29.164925 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:33.832049 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:33.832959 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:33.832959 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:33.834109 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:33.878964 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:33.879447 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:33.880687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:33.880687 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:33.880687 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:33.880687 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:33.880687 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:33.880687 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:33.880687 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:33.880687 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:33.882133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.048 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:33.882133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:33.882133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:33.882133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:33.882133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:33.882133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:47:33.882133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:33.882133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:33.882133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:33.882133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:33.882133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:33.882133 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:33.882384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:33.882384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:33.882384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:33.882384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:33.882384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:33.882384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:33.882384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:33.882384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:33.882384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:33.882384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:33.882384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:33.882384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:33.882384 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:33.882618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:33.882618 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:33.883329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:33.883329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:33.883329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:33.883329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:33.883329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:33.883329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:33.883329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:33.883329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:33.883329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:33.883329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:33.883329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:33.883329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:33.883329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:33.883329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:33.883329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:33.883329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:33.883654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:47:33.883654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:33.883654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:33.883654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:33.883654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:33.883654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:33.883654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:33.883654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:33.883654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:33.883654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:33.883654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:33.883654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:33.883654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:33.883654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:33.883654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:33.883654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:33.883654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:33.883654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:33.883654 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:33.884020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:33.884020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:33.884020 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:34.046595 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:34.047123 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:34.047348 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:34.048579 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:34.048971 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:34.049489 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.050162 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:34.050162 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:34.050162 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:34.050162 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:34.050162 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:34.050162 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:34.050162 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.050162 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:34.051866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.051866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:34.051866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:34.051866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:34.051866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:34.051866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:47:34.051866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:34.051866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:34.051866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:34.051866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:34.051866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:34.051866 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:34.052410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:34.052410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:34.052410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:34.052410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:34.052410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:34.052410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:34.052410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:34.052410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:34.052410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:34.052410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:34.052410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:34.052410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:34.052410 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:34.052910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.052910 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:34.053642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.053642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:34.053642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:34.053642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:34.053642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:34.053642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:34.053642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:34.053642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:34.053642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:34.053642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:34.053642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:34.053642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:34.053642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:34.053642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:34.053642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:34.053642 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:34.054580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:47:34.054580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:34.054580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:34.054580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:34.054580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:34.054580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:34.054580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:34.054580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:34.054580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:34.054580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:34.054580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:34.054580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:34.054580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:34.054580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:34.054580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:34.054580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:34.054580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:34.054580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:34.054580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:34.055639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:34.055639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.055639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:34.055639 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:34.055639 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:34.055639 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:34.055639 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:47:34.056008 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:34.056008 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.056733 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:34.056733 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:34.056733 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:34.056733 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:34.056733 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:34.056733 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:34.056733 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.056733 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:34.057434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.057434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:47:34.057434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:34.057434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:34.057434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:34.057434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:34.057434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:34.057434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:34.057434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:34.057434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:34.057434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:34.057434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:47:34.057434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:34.057434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:47:34.057434 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:47:34.057851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:34.057851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:47:34.057851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:34.057851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:47:34.057851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:47:34.057851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:34.057851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:47:34.057851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.057851 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:47:34.058355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.058355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:34.058355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:34.058355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:34.058355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:34.058355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:47:34.058355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:47:34.058355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:47:34.058355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:34.058355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:34.058355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:34.058355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:34.059082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:34.059082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:34.059082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:34.059082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:34.059082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:34.059082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:34.059082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:34.059082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:34.059082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:34.059082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:34.059082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:34.059082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:34.059082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:34.059082 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:34.059951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:34.059951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:34.059951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.059951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:34.059951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.059951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:34.059951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:34.059951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:34.059951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:34.059951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:34.059951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:34.059951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:34.059951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:34.059951 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:34.060334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:34.060334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:34.060334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:34.060334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:34.060334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:34.060334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:34.060334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:47:34.060334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:47:34.060334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:47:34.060334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:34.060334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:34.060334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:34.060334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:34.060334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:34.060334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:34.060334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:34.060334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:34.060334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:34.060334 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:34.060902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:34.060902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:34.060902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:34.060902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:34.060902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:34.060902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:34.060902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:34.060902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:34.060902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:34.060902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:34.060902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.060902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:34.077182 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:34.077739 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:34.077951 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:34.078167 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:34.078783 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:34.079146 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.079972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:34.079972 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:34.079972 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:34.079972 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:34.079972 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:34.079972 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:34.079972 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.079972 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:34.081329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.081329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:47:34.081329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:34.081329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:34.081329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:34.081329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:34.081329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:34.081329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:34.081329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:34.081329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:34.081329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:34.081329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:47:34.081329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:34.081329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:47:34.081329 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:47:34.081762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:34.081762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:47:34.081762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:34.081762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:47:34.081762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:47:34.081762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:34.081762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:47:34.081762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.081762 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:47:34.082892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.082892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:34.082892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:34.082892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:34.082892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:34.082892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:47:34.082892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:47:34.082892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:34.082892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:34.082892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:34.082892 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:34.083252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:34.083252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:34.083252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:34.083252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:34.083252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:34.083252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:34.083252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:34.083252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:34.083252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:34.083252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:34.083252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:34.083252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:34.083252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:34.083252 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:34.083669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:34.083669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:34.083669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.083669 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:34.084706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.084706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:34.084706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:34.084706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:34.084706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:34.084706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:34.084706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:34.084706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:34.084706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:34.084706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:34.084706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:34.084706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:34.084706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:34.084706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:34.084706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:34.084706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:34.084706 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:47:34.085192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:47:34.085192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:34.085192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:34.085192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:34.085192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:34.085192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:34.085192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:34.085192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:34.085192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:34.085192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:34.085192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:34.085192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:34.085192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:34.085192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:34.085192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:34.085192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:34.085192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:34.085192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:34.085192 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:34.085722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:34.085722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:34.085722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.085722 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:34.157443 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:34.158159 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:34.158408 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:34.158808 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:34.159410 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:34.159863 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:34.160097 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.161065 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:34.161065 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:34.161065 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:34.161065 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:34.161065 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:34.161065 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:34.161065 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.161065 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:34.162514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.162514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:34.162514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:34.162514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:34.162514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:34.162514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:47:34.162514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:34.162514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:34.162514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:34.162514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:34.162514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:34.162514 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:34.163119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:34.163119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:34.163119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:34.163119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:34.163119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:34.163119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:34.163119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:34.163119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:34.163119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:34.163119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:34.163119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:34.163119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:34.163119 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:34.163670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.163670 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:34.164063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.164063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:34.164063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:34.164063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:34.164063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:34.164063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:34.164063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:34.164063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:34.164063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:34.164063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:34.164063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:34.164063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:34.164063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:34.164063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:34.164063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:34.164063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:34.164751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:47:34.164751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:34.164751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:34.164751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:34.164751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:34.164751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:34.164751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:34.164751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:34.164751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:34.164751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:34.164751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:34.164751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:34.164751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:34.164751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:34.164751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:34.164751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:34.164751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:34.164751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:34.164751 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:34.165547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:34.165547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:34.165547 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:38.832934 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:38.833837 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:38.833837 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:38.835272 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:38.866281 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:38.866659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:38.867588 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:38.867588 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:38.867588 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:38.867588 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:38.867588 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:38.867588 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:38.867588 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:38.867588 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:38.868428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:38.868428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:38.868428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:38.868428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:38.868428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:38.868428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:47:38.868428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:38.868428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:38.868428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:38.868428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:38.868428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:38.868428 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:38.869203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:38.869203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:38.869203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:38.869203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:38.869203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:38.869203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:38.869203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:38.869203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:38.869203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:38.869203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:38.869203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:38.869203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:38.869203 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:38.869961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:38.869961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:38.869961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:38.869961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:38.869961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:38.869961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:38.869961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:38.869961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:38.869961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:38.869961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:38.869961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:38.869961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:38.869961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:38.869961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:38.869961 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:38.871568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:38.871568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:38.871568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:38.871568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:47:38.871568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:38.871568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:38.871568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:38.871568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:38.871568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:38.871568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:38.871568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:38.871568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:38.871568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:38.871568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:38.871568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:38.871568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:38.871568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:38.871568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:38.871568 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:38.873173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:38.873173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:38.873173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:38.873173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:38.873173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:38.873173 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:39.047435 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:39.047913 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:39.048159 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:39.049324 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:39.049698 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:39.049983 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.050902 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:39.050902 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:39.050902 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:39.050902 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:39.050902 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:39.050902 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:39.050902 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.050902 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:39.052580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.052580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:39.052580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:39.052580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:39.052580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:39.052580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:47:39.052580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:39.052580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:39.052580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:39.052580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:39.052580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:39.052580 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:39.053346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:39.053346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:39.053346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:39.053346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:39.053346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:39.053346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:39.053346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:39.053346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:39.053346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:39.053346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:39.053346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:39.053346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:39.053346 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:39.054057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.054057 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:39.054377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.054377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:39.054377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:39.054377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:39.054377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:39.054377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:39.054377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:39.054377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:39.054377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:39.054377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:39.054377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:39.054377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:39.054377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:39.054377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:39.054377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:39.054377 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:39.055193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:47:39.055193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:39.055193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:39.055193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:39.055193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:39.055193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:39.055193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:39.055193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:39.055193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:39.055193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:39.055193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:39.055193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:39.055193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:39.055193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:39.055193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:39.055193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:39.055193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:39.055193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:39.055193 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:39.056256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:39.056256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.056256 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:39.056256 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:39.056256 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:39.056256 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:39.056256 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:47:39.056847 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:39.056847 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.057659 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:39.057659 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:39.057659 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:39.057659 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:39.057659 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:39.057659 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:39.057659 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.057659 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:39.058629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.058629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:47:39.058629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:39.058629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:39.058629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:39.058629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:39.058629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:39.058629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:39.058629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:39.058629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:39.058629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:39.058629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:47:39.058629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:39.058629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:47:39.058629 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:47:39.059480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:39.059480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:47:39.059480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:39.059480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:47:39.059480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:47:39.059480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:39.059480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:47:39.059480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.059480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:47:39.059480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.059480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:39.059480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:39.059480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:39.059480 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:39.060321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:47:39.060321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:47:39.060321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:47:39.060321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:39.060321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:39.060321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:39.060321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:39.060321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:39.060321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:39.060321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:39.060321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:39.060321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:39.060321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:39.061111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:39.061111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:39.061111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:39.061111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:39.061111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:39.061111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:39.061111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:39.061111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:39.061111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:39.061111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:39.061111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.061111 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:39.061736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.061736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:39.061736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:39.061736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:39.061736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:39.061736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:39.061736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:39.061736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:39.061736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:39.061736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:39.061736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:39.061736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:39.061736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:39.061736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:39.061736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:39.061736 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:39.062501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:47:39.062501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:47:39.062501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:47:39.062501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:39.062501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:39.062501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:39.062501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:39.062501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:39.062501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:39.062501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:39.062501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:39.062501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:39.062501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:39.062501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:39.062501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:39.062501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:39.062501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:39.062501 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:39.063442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:39.063442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:39.063442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:39.063442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:39.063442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:39.063442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.063442 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:39.077782 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:39.078228 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:39.078410 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:39.078605 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:39.079187 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:39.079519 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.080321 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:39.080321 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:39.080321 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:39.080321 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:39.080321 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:39.080321 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:39.080321 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.080321 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:39.082144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.082144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:47:39.082144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:39.082144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:39.082144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:39.082144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:39.082144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:39.082144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:39.082144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:39.082144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:39.082144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:39.082144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:47:39.082144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:39.082144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:47:39.082144 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:47:39.083080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:39.083080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:47:39.083080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:39.083080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:47:39.083080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:47:39.083080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:39.083080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:47:39.083080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.083080 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:47:39.084107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.084107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:39.084107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:39.084107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:39.084107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:39.084107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:47:39.084107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:47:39.084107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:39.084107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:39.084107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:39.084107 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:39.084717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:39.084717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:39.084717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:39.084717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:39.084717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:39.084717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:39.084717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:39.084717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:39.084717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:39.084717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:39.084717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:39.084717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:39.084717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:39.084717 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:39.085355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:39.085355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:39.085355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.085355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:39.085355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.085355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:39.085355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:39.085355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:39.085355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:39.085355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:39.085355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:39.085355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:39.085355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:39.085355 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:39.086041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:39.086041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:39.086041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:39.086041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:39.086041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:39.086041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:39.086041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:47:39.086041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:47:39.086041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:39.086041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:39.086041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:39.086041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:39.086041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:39.086041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:39.086041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:39.086041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:39.086041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:39.086041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:39.086041 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:39.086958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:39.086958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:39.086958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:39.086958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:39.086958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:39.086958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:39.086958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:39.086958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:39.086958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:39.086958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.086958 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:39.158223 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:39.158898 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:39.159182 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:39.159480 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:39.160137 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:39.160502 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:39.160776 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.161608 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:39.161608 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:39.161608 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:39.161608 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:39.161608 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:39.161608 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:39.161608 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.161608 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:39.163320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.163320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:39.163320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:39.163320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:39.163320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:39.163320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:47:39.163320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:39.163320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:39.163320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:39.163320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:39.163320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:39.163320 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:39.163802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:39.163802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:39.163802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:39.163802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:39.163802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:39.163802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:39.163802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:39.163802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:39.163802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:39.163802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:39.163802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:39.163802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:39.163802 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:39.164212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.164212 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:39.164662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.164662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:39.164662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:39.164662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:39.164662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:39.164662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:39.164662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:39.164662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:39.164662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:39.164662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:39.164662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:39.164662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:39.164662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:39.164662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:39.164662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:39.164662 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:39.165297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:47:39.165297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:39.165297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:39.165297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:39.165297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:39.165297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:39.165297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:39.165297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:39.165297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:39.165297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:39.165297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:39.165297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:39.165297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:39.165297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:39.165297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:39.165297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:39.165297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:39.165297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:39.165297 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:39.166063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:39.166063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:39.166063 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:43.833615 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:43.834441 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_fdb_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:43.834441 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply FDB aging limits to NB_Global and provider switches. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:43.835114 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'fdb_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:43.866812 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'fdb_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:43.867185 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:43.868058 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:43.868058 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:43.868058 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:43.868058 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:43.868058 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:43.868058 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:43.868058 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:43.868058 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:43.868846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_fdb_aging_settings failed after 0.034 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:43.868846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:43.868846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:43.868846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:43.868846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:43.868846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:47:43.868846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:43.868846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:43.868846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:43.868846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:43.868846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:43.868846 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:43.869356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:43.869356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:43.869356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:43.869356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:43.869356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:43.869356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:43.869356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:43.869356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:43.869356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:43.869356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:43.869356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:43.869356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:43.869356 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:43.869990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:43.869990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:43.869990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_fdb_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:43.869990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:43.869990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:43.869990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:43.869990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:43.869990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:43.869990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:43.869990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:43.869990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:43.869990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:43.869990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:43.869990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:43.869990 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:43.870687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:43.870687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:43.870687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:43.870687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 884, in check_fdb_aging_settings
Apr 28 20:47:43.870687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:43.870687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:43.870687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:43.870687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:43.870687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:43.870687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:43.870687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:43.870687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:43.870687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:43.870687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:43.870687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:43.870687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:43.870687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:43.870687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:43.870687 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:43.871479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:43.871479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:43.871479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:43.871479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:43.871479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:43.871479 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:44.048044 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:44.048565 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: check_router_default_route_empty_dst_ip {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:44.048779 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Remove default routes with empty destination from routers. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:44.049756 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelStaticRoutesCommand(_result=None, lrouter=neutron-15fdd774-e739-4763-a086-bc4b40fb7431, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:44.049965 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DelStaticRoutesCommand(_result=None, lrouter=neutron-6feb526a-13c6-41ac-983d-ea1d2251ff3d, routes=[], if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:44.050088 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.050656 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:44.050656 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:44.050656 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:44.050656 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:44.050656 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:44.050656 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:44.050656 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.050656 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:44.052421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task check_router_default_route_empty_dst_ip failed after 0.002 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.052421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:44.052421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:44.052421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:44.052421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:44.052421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:47:44.052421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:44.052421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:44.052421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:44.052421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:44.052421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:44.052421 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:44.053027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:44.053027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:44.053027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:44.053027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:44.053027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:44.053027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:44.053027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:44.053027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:44.053027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:44.053027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:44.053027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:44.053027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:44.053027 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:44.053694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.053694 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:44.054388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_router_default_route_empty_dst_ip' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.054388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:44.054388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:44.054388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:44.054388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:44.054388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:44.054388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:44.054388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:44.054388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:44.054388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:44.054388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:44.054388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:44.054388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:44.054388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:44.054388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:44.054388 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:44.054843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 984, in check_router_default_route_empty_dst_ip
Apr 28 20:47:44.054843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:44.054843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:44.054843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:44.054843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:44.054843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:44.054843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:44.054843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:44.054843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:44.054843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:44.054843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:44.054843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:44.054843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:44.054843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:44.054843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:44.054843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:44.054843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:44.054843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:44.054843 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:44.055393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:44.055393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.055393 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:44.055559 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:44.056010 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: set_fip_distributed_flag {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:44.056193 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Set distributed floating IP flag in NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:44.056322 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting fip-distributed flag in NB_Global to False {{(pid=106506) set_fip_distributed_flag /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:1208}}
Apr 28 20:47:44.057188 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('external_ids', {'neutron:fip-distributed': 'False'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:44.057639 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.058661 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:44.058661 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:44.058661 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:44.058661 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:44.058661 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:44.058661 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:44.058661 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.058661 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:44.059408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.059408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:47:44.059408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:44.059408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:44.059408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:44.059408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:44.059408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:44.059408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:44.059408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:44.059408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:44.059408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:44.059408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:47:44.059408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:44.059408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:47:44.059408 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:47:44.059868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:44.059868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:47:44.059868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:44.059868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:47:44.059868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:47:44.059868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:44.059868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:47:44.059868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.059868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:47:44.060291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task set_fip_distributed_flag failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.060291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:44.060291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:44.060291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:44.060291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:44.060291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:47:44.060291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:47:44.060291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                                                           ^^^^^^^^
Apr 28 20:47:44.060291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:44.060291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:44.060291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:44.060291 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:44.060617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:44.060617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:44.060617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:44.060617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:44.060617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:44.060617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:44.060617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:44.060617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:44.060617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:44.060617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:44.060617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:44.060617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:44.060617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:44.060617 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:44.061113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:44.061113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:44.061113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.061113 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:44.061341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.set_fip_distributed_flag' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.061341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:44.061341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:44.061341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:44.061341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:44.061341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:44.061341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:44.061341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:44.061341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:44.061341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:44.061341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:44.061341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:44.061341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:44.061341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:44.061341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:44.061341 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:44.061933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1212, in set_fip_distributed_flag
Apr 28 20:47:44.061933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     ovn_const.OVN_FIP_DISTRIBUTED_KEY: str(distributed)}).execute(
Apr 28 20:47:44.061933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                                                           ^^^^^^^^
Apr 28 20:47:44.061933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:44.061933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:44.061933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:44.061933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:44.061933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:44.061933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:44.061933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:44.061933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:44.061933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:44.061933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:44.061933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:44.061933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:44.061933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:44.061933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:44.061933 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:44.062460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:44.062460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:44.062460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:44.062460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:44.062460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:44.062460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.062460 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:44.078316 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:44.078670 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_ha_failover {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:44.078916 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Apply HA chassis failover BFD settings to NB_Global. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:44.079104 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:44.079749 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'bfd-min-rx': '1000', 'bfd-min-tx': '100', 'bfd-mult': '3'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:44.080247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.081350 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:44.081350 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:44.081350 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:44.081350 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:44.081350 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:44.081350 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:44.081350 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.081350 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:44.082671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Error executing command (DbSetCommand): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.082671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command Traceback (most recent call last):
Apr 28 20:47:44.082671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:44.082671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:44.082671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:44.082671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:44.082671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:44.082671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:44.082671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:44.082671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     next(self.gen)
Apr 28 20:47:44.082671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:44.082671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     with self.create_transaction(
Apr 28 20:47:44.082671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:44.082671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     self.result = self.commit()
Apr 28 20:47:44.082671 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                   ^^^^^^^^^^^^^
Apr 28 20:47:44.083602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:44.083602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise result.ex
Apr 28 20:47:44.083602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:44.083602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     txn.results.put(txn.do_commit())
Apr 28 20:47:44.083602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command                     ^^^^^^^^^^^^^^^
Apr 28 20:47:44.083602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:44.083602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command     raise RuntimeError(msg)
Apr 28 20:47:44.083602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.083602 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.command 
Apr 28 20:47:44.084247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_ha_failover failed after 0.004 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.084247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:44.084247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:44.084247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:44.084247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:44.084247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:47:44.084247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:47:44.084247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:44.084247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:44.084247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:44.084247 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:44.084972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:44.084972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:44.084972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:44.084972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:44.084972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:44.084972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:44.084972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:44.084972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:44.084972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:44.084972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:44.084972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:44.084972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:44.084972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:44.084972 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:44.085451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:44.085451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:44.085451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.085451 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:44.085873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_ha_failover' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.085873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:44.085873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:44.085873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:44.085873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:44.085873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:44.085873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:44.085873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:44.085873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:44.085873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:44.085873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:44.085873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:44.085873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:44.085873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:44.085873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:44.085873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:44.085873 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 1273, in update_ha_failover
Apr 28 20:47:44.086360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self._nb_idl.set_nb_global_options(**options).execute(
Apr 28 20:47:44.086360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/command.py", line 49, in execute
Apr 28 20:47:44.086360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.api.transaction(check_error, log_errors, **kwargs) as t:
Apr 28 20:47:44.086360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:44.086360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:44.086360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:44.086360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:44.086360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:44.086360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:44.086360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:44.086360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:44.086360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:44.086360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:44.086360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:44.086360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:44.086360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:44.086360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:44.086360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:44.086360 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:44.086949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:44.086949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:44.086949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.086949 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics 
Apr 28 20:47:44.158764 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' {{(pid=106506) _process_scheduled /opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py:784}}
Apr 28 20:47:44.159167 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Starting OVN maintenance task: update_mac_aging_settings {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:99}}
Apr 28 20:47:44.159395 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task: Update MAC binding aging and router MAC age limits. {{(pid=106506) wrapper /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:101}}
Apr 28 20:47:44.159725 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.impl_idl_ovn [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Setting NB_Global options: {'mac_binding_removal_limit': '0'} {{(pid=106506) set_nb_global_options /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py:950}}
Apr 28 20:47:44.160411 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(_result=None, table=NB_Global, record=., col_values=dict_items([('options', {'mac_binding_removal_limit': '0'})]), if_exists=True) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:44.160986 np0000171185 neutron-ovn-maintenance-worker[106506]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): SetLRouterMacAgeLimitCommand(_result=None, router=None, threshold=300) {{(pid=106506) do_commit /opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Apr 28 20:47:44.161284 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [-] OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.162238 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR ovsdbapp.backend.ovs_idl.transaction [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Traceback (most recent call last):
Apr 28 20:47:44.162238 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:44.162238 np0000171185 neutron-ovn-maintenance-worker[106506]:     txn.results.put(txn.do_commit())
Apr 28 20:47:44.162238 np0000171185 neutron-ovn-maintenance-worker[106506]:                     ^^^^^^^^^^^^^^^
Apr 28 20:47:44.162238 np0000171185 neutron-ovn-maintenance-worker[106506]:   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:44.162238 np0000171185 neutron-ovn-maintenance-worker[106506]:     raise RuntimeError(msg)
Apr 28 20:47:44.162238 np0000171185 neutron-ovn-maintenance-worker[106506]: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.162238 np0000171185 neutron-ovn-maintenance-worker[106506]: 
Apr 28 20:47:44.163237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] OVN maintenance task update_mac_aging_settings failed after 0.003 seconds: RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.163237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance Traceback (most recent call last):
Apr 28 20:47:44.163237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:44.163237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     result = f(*args, **kwargs)
Apr 28 20:47:44.163237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:44.163237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:47:44.163237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:44.163237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:44.163237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:44.163237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:44.163237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:44.163237 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:44.164127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     next(self.gen)
Apr 28 20:47:44.164127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:44.164127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     with self.create_transaction(
Apr 28 20:47:44.164127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:44.164127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     self.result = self.commit()
Apr 28 20:47:44.164127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                   ^^^^^^^^^^^^^
Apr 28 20:47:44.164127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:44.164127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise result.ex
Apr 28 20:47:44.164127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:44.164127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     txn.results.put(txn.do_commit())
Apr 28 20:47:44.164127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance                     ^^^^^^^^^^^^^^^
Apr 28 20:47:44.164127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:44.164127 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance     raise RuntimeError(msg)
Apr 28 20:47:44.164982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.164982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance 
Apr 28 20:47:44.164982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics [None req-2b41a414-6d5d-40e1-94a9-4bd5c9ebf662 None None] Failed to call periodic 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.update_mac_aging_settings' (it runs every 5.00 seconds): RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.164982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics Traceback (most recent call last):
Apr 28 20:47:44.164982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 376, in run
Apr 28 20:47:44.164982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     work()
Apr 28 20:47:44.164982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 90, in __call__
Apr 28 20:47:44.164982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return self.callback(*self.args, **self.kwargs)
Apr 28 20:47:44.164982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:44.164982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/futurist/periodics.py", line 213, in decorator
Apr 28 20:47:44.164982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(*args, **kwargs)
Apr 28 20:47:44.164982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:44.164982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 81, in decorator
Apr 28 20:47:44.164982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     return f(self, *args, **kwargs)
Apr 28 20:47:44.164982 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics            ^^^^^^^^^^^^^^^^^^^^^^^^
Apr 28 20:47:44.165868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 107, in wrapper
Apr 28 20:47:44.165868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     result = f(*args, **kwargs)
Apr 28 20:47:44.165868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics              ^^^^^^^^^^^^^^^^^^
Apr 28 20:47:44.165868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py", line 900, in update_mac_aging_settings
Apr 28 20:47:44.165868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self._nb_idl.transaction(check_error=True) as txn:
Apr 28 20:47:44.165868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:44.165868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:44.165868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 271, in transaction
Apr 28 20:47:44.165868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with super().transaction(*args, **kwargs) as t:
Apr 28 20:47:44.165868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
Apr 28 20:47:44.165868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     next(self.gen)
Apr 28 20:47:44.165868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 114, in transaction
Apr 28 20:47:44.165868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     with self.create_transaction(
Apr 28 20:47:44.165868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/api.py", line 71, in __exit__
Apr 28 20:47:44.165868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     self.result = self.commit()
Apr 28 20:47:44.165868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                   ^^^^^^^^^^^^^
Apr 28 20:47:44.165868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 64, in commit
Apr 28 20:47:44.165868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise result.ex
Apr 28 20:47:44.165868 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 127, in run
Apr 28 20:47:44.166814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     txn.results.put(txn.do_commit())
Apr 28 20:47:44.166814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics                     ^^^^^^^^^^^^^^^
Apr 28 20:47:44.166814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics   File "/opt/stack/data/venv/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
Apr 28 20:47:44.166814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics     raise RuntimeError(msg)
Apr 28 20:47:44.166814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics RuntimeError: OVSDB Error: The transaction failed because the IDL has been configured to require a database lock but didn't get it yet or has already lost it
Apr 28 20:47:44.166814 np0000171185 neutron-ovn-maintenance-worker[106506]: ERROR futurist.periodics